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Preface 


The  United  States  has  an  important  policy  interest  in  training  foreign  nationals  in  English 
because  the  English  language  serves  as  a  foundation  for  aviation  and  international  military 
operations  and  because  much  of  the  information  highway  is  in  English.  U.S.  operations 
with  foreign  nationals  are  greatly  enhanced  if  those  foreign  nationals  understand  English. 
The  Defense  Language  Institute  English  Language  Center  (DLIELC)  trains  foreign  nation¬ 
als  in  English  prior  to  their  attending  U.S.  military  education  and  training  courses  and  plays 
a  critical  role  in  building  partnerships.  RAND  was  asked  to  evaluate  demand,  supply,  pricing 
models,  and  other  options  to  optimize  DLIELC’s  output. 

The  research  described  in  this  report  extended  over  five  months,  from  September  2010 
through  January  2011. 

The  study  was  sponsored  by  the  Air  Force  Language,  Region  and  Culture  Program  Office, 
Policy  (AF/A1DG),  and  the  Office  of  the  Secretary  of  Defense  for  Personnel  and  Readiness, 
Defense  Language  Office  (OSD  (P&R)/(DLO)).  The  research  was  conducted  within  the  Man¬ 
power,  Personnel,  and  Training  Program  of  RAND  Project  AIR  FORCE  as  part  of  a  fiscal  year 
2010-2011  study,  “Optimizing  the  Defense  Language  Institute  English  Language  Center  to 
Meet  Current  and  Future  Requirements.” 

This  report  should  be  of  interest  to  military  leaders  and  staffs  concerned  with  increasing 
the  effectiveness  of  English  language  training  of  foreign  military  personnel  and  those  respon¬ 
sible  for  programs  related  to  building  partnerships  with  other  nations’  militaries. 


RAND  Project  AIR  FORCE 

RAND  Project  AIR  FORCE  (PAF),  a  division  of  the  RAND  Corporation,  is  the  U.S.  Air 
Force’s  federally  funded  research  and  development  center  for  studies  and  analyses.  PAF  pro¬ 
vides  the  Air  Force  with  independent  analyses  of  policy  alternatives  affecting  the  development, 
employment,  combat  readiness,  and  support  of  current  and  future  air,  space,  and  cyber  forces. 
Research  is  conducted  in  four  programs:  Force  Modernization  and  Employment;  Manpower, 
Personnel,  and  Training;  Resource  Management;  and  Strategy  and  Doctrine. 

Additional  information  about  PAF  is  available  on  our  website: 
http :  //  www.  rand,  org/ p  af / 
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Summary 


Introduction 

The  Defense  Language  Institute  English  Language  Center  (DLIELC)  plays  an  important  role 
in  the  broad  spectrum  of  the  U.S.  Department  of  Defense’s  (DoD’s)  security  cooperation 
objectives  as  a  key  enabler  and  strategic  engagement  tool  by  training  U.S.  military  and  for¬ 
eign  students  in  English.  However,  the  U.S.  Air  Force  (USAF) — which  is  the  organization’s 
executive  agent — needs  a  better  understanding  of  DLIELC’s  cost  structure  and  a  capability  to 
measure  the  mid-  and  long-term  demand1  for  English  language  training  (ELT)  across  the  DoD 
and  foreign  partners. 

In  September  2010,  the  Air  Force  Language,  Region  and  Culture  Program  Office,  Policy 
(AF/A1DG),  and  the  Office  of  the  Secretary  of  Defense  for  Personnel  and  Readiness,  Defense 
Language  Office  (OSD(P&R)/(DLO))  asked  PAF  to  examine  DLIELC  to  ensure  that  it  is 
appropriately  tasked,  organized,  operated,  resourced,  and  managed  to  produce  the  ELT  capa¬ 
bility  needed  to  effectively  and  efficiently  meet  DoD  mission  requirements. 

To  accomplish  those  objectives,  the  study  included  four  broad  tasks:  (1)  review  prior 
studies,  authorities,  and  governing  documents  pertaining  to  DLIELC;  (2)  develop  a  require¬ 
ments  process  to  identify  and  predict  all  the  sources  of  student  throughput,  in  sufficient  detail 
to  program  resource  needs;  (3)  consider  options  to  improve  the  organization,  reporting,  and 
resourcing  structure;  and  (4)  assist  DLIELC  in  its  strategic  planning. 

To  accomplish  these  tasks,  we  relied  on  student  data  from  DLIELC  from  2001  to  2010. 
To  gather  data  from  DLIELC,  we  used  a  question-and-answer  process  in  which  the  team  sub¬ 
mitted  a  question  to  a  designated  DLIELC  official,  the  question  was  then  researched,  and  the 
answer  and  accompanying  data  were  provided  to  the  study  team  in  a  timely  manner.  In  addi¬ 
tion,  we  interviewed  relevant  officials.  To  identify  best  practices,  we  also  reviewed  pertinent 
regulations,  directives,  instructions,  and  policy  documents.  Finally,  we  conducted  a  compara¬ 
tive  analysis  with  12  similar  organizations  that  had  a  security  cooperation  focus. 


Key  Recommendations 

Based  on  our  analysis,  we  identified  problems  in  eight  critical  management  areas,  which  can  be 
thought  of  in  terms  of  the  elements  of  effective  organizations  to  which  they  correspond: 


1  Mid-term  demand  is  defined  as  Future  Years  Defense  Program  (FYDP)  years.  Long-term  demand  would  start  near  the 
end  of  the  FYDP  and  continue  beyond. 
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1.  Policy.  DLIELC  is  governed  by  a  maze  of  guidance.  Additionally,  that  guidance  does 
not  necessarily  reflect  the  reality  of  the  circumstances  within  which  it  must  operate  and 
consequently  inhibits  effective  oversight. 

2.  Business  Model/Requirements.  DLIELC  lacks  a  robust  requirements  determination 
process,  and  the  organization’s  business  model  is  unable  to  meet  variable  demand. 

3.  Financial  Risk.  Current  financial  management  practices  expose  ELT  clients  to  need¬ 
less  financial  risk. 

4.  Technology.  DLIELC  should  introduce  new  technologies  at  a  quicker  rate;  their  suc¬ 
cessful  use  could  lower  fixed  and  variable  costs. 

5.  Identity.  DLIELC  has  conflicting  priorities  that  have  led  to  contradictory  views  on  the 
institution’s  identity  (i.e.,  is  it  an  academic,  military,  or  government/policy  institution?). 

6.  Assessment.  The  lack  of  a  formal  and  functioning  assessment/evaluation  process  inhib¬ 
its  DLIELC  from  being  able  to  define  and  assess  its  own  performance. 

7.  Manpower.  The  hiring  process  is  unable  to  respond  quickly  to  short-term  and  cyclical 
demand. 

8.  Organization/Advocacy.  Importantly  the  organization  has  had  no  clear  institutional 
advocate,  resulting  in  a  lack  of  focused  oversight  within  the  DoD  as  it  relates  to  ensur¬ 
ing  the  effective  and  efficient  execution  of  ELT. 

Our  recommendations  in  each  of  these  eight  areas  are  listed  in  Table  S.l. 


Implementation 

Implementing  changes  in  the  above  management  areas  will  not  only  promote  organizational 
effectiveness  but  will  also  help  to  ensure  that  DLIELC  is  provided  with  the  resources  it  needs 
to  fulfill  its  mission.  An  implementation  plan  is  needed  to  integrate  all  the  recommendations 
in  a  way  that  informs  the  leadership  of  Headquarters  Air  Education  and  Training  Command 
(HQ  AETC)  and  the  DLIELC  management  of  the  implications  of  change  throughout  the 
organization  (the  matrix  in  Appendix  C  is  provided  for  this  purpose). 

The  implementation  plan,  shown  in  Table  S.2,  translates  the  study’s  key  recommenda¬ 
tions  into  13  general  outcomes.  The  key  organization  for  most  outcomes  is  DLIELC,  but  other 
agencies — the  37th  Training  Wing,  HQ  AETC,  AF/A1DG,  the  Defense  Security  Coopera¬ 
tion  Agency  (DSCA),  and  the  Office  of  the  Secretary  of  Defense  (OSD) — are  responsible  for 
specific  inputs  to  achieve  the  outcomes.  Chapter  Ten  discusses  the  implementation  plan  in 
more  detail,  while  Appendix  C  provides  a  guiding  matrix.  We  summarize  the  13  key  outcomes 
in  terms  of  the  critical  area  and  priority  in  Table  S.2. 
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Table  S.1 

Recommendations 


Critical  Area 

Recommendations 

Policy 

Fully  review  all  applicable  rules  and  regulations  to  ensure  currency  and  appropriateness 

Create  two  instructions,  a  governing  DoD  instruction  and  a  consolidated  AF  instruction, 
covering  everything  needed  to  grade,  audit,  govern,  cost,  and  direct  the  organization 

Consider  leveraging  existing  AETC  processes  and  procedures  where  they  may  create 
opportunities  to  fully  tap  into  AETC's  resources  and  expertise 

Review  existing  rules  and  regulations  to  look  for  underutilized  authorities  and  provisions  that 
could  be  exercised  to  enhance  DLlELC's  effectiveness 

Business  model/ 
requirements 

Institute  a  process  to  enable  more  accurate  depiction  of  demand  using  the  proposed  model 
Reduce  seasonal  peaks  by  delaying  certain  categories  of  students 

Improve  utilization  of  existing  instructors  by  restricting  leave  during  peak  season 

Create  "breathing  space"  within  existing  supply  via  possible  relocation  of  U.S.  Army  students 
or  by  the  creation  of  a  "finishing  class" 

Increase  supply  by  utilizing  proposed  supply  strategy 

Financial  risk 

Form  a  task  force  of  senior  leadership  from  the  37th  Training  Wing,  HQ  AETC,  and  DLIELC  to 
implement  cost  recovery  reform 

Flave  the  task  force  review  proposed  costing  models  to  ensure  methodological  accuracy 

Once  validated,  fully  cost  DLlELC's  activities 

After  costing  library  is  developed,  review  and  reissue  financial  management  policies  and 
procedures  as  they  relate  to  DLIELC 

Technology 

Explore  where  there  may  be  cost-effective  means  to  deliver  some  ELT  through  distance 
learning 

Explore  computerized  methods,  such  as  OPIc® 

Consider  using  a  collaborative  platform  approach  to  update  SET  textbooks 

Provide  all  students  (not  just  SET  students)  with  laptops 

Buy  additional  commercial  bandwidth  for  Internet  usage;  add  costs  to  fixed  cost  of  operating 
schoolhouse 

Identity 

Deemphasize  the  academic  character  of  the  organization 

Look  beyond  ELT  credentials  in  teacher  recruitment  process 

Have  more  military  personnel  interact  with  IMSs 

Consider  creating  an  alumni  program 

Consider  involving  IMSs  in  self-assessment  for  placement,  progression,  or  curriculum 
Collaborate  with  Defense  Language  Testing  Advisory  Board  to  fully  review  the  validity  and 
reliability  of  the  ECL 

Expand  faculty  PST  to  provide  basic  skills  and  understanding  in  security  cooperation 
Reemphasize  and  develop  field  trips  and  cultural  activities 

Assessment 

Leverage  AETC's  existing  training  assessment  process  including  conformance  to  training 
development  and  evaluation  policies 

Explore  standard  processes  for  training  development  and  assessment,  gaining  access  to  lessons 
from  similar  training  organizations 

Improve  assessment  of  the  FSP  and  coordinate  with  the  Air  Force  Culture  and  Language  Center 
to  develop  measures  for  assessing  knowledge  and  comprehension  of  U.S.  culture 

Reexamine  balanced  scorecard  to  fully  represent  each  of  the  four  performance  measure 
perspectives 

Reevaluate  current  metrics  to  ensure  they  are  clearly  written,  relevant,  inclusive  of  academic, 
military,  and  building  partnerships  (BP)  objectives,  and  primarily  under  DLlELC's  control 

Include  specific  metrics  geared  toward  measuring  the  long-term  BP  impact 
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Table  S.1 — Continued 


Critical  Area  Recommendations 

Manpower  Utilize  U.S.  Code,  Title  10,  civilian  academic  hires  provisions 

End  the  current  practice  of  curriculum  development,  which  unnecessarily  consumes  faculty 
manpower 

Establish  a  dedicated  contracting  vehicle  to  purchase  English  language  instruction  and  support 

Have  specialized  military/technical  language  curricula  requirements  executed  by 
knowledgeable  experts  in  those  fields 

Emphasize  the  importance  of  building  partnerships  by  obtaining  more  military  department 
personnel 

Use  experts  and  consultants  as  a  ready  pool  of  language  professionals  who  could  be  brought 
on  to  quickly  fill  gaps  because  of  unforeseen  student  load  fluctuations 


Organization/  Ensure  that  OSD  and  USAF  leadership  agree  on  clarified  oversight  responsibilities  and 
advocacy  functions  for  DLIELC 

Ensure  that  an  advocate  or  champion  in  DoD  is  actively  involved  in  the  requirements  process 

Fully  implement  principles,  practices,  and  norms  established  by  AETC  to  ensure  DLIELC 
becomes  more  responsive  and  accountable  to  DoD  policies  and  priorities 


NOTES:  SET  =  Specialized  English  training;  IMS  =  international  military  student. 


Table  S.2 

Recommendations  to  Address  Problems  in  the  Critical  Areas  Identified 


Outcome  (Critical  Area) 

1.  Clarified  mission  statement/priorities  in  conformance  with  USAF,  DSCA,  OSD  policies  and 
priorities  (M/P;  O/A) 

2.  Enhanced  policies/procedures,  implemented  to  increase  managerial  effectiveness  (M/P;  O/A) 

3.  Policy  to  improve  prediction  of  ELT  demand  (R;  I;  M) 

4.  Policy  to  prioritize  student  flow  by  reducing  seasonal  demand  peaks  (R) 

5.  Policy  that  expands  teaching  capacity  by  existing  labor  supply  and  that  introduces  greater 
flexibility  in  managing  labor  to  meet  peak  demand  periods  (FM/M) 

6.  Policy  to  manage  increase  in  supply  of  ELT  (R) 

7.  Policy  that  overhauls  existing  management  of  finance  system,  practices,  and  key  supporting 
tools  (MP;  FM) 

8.  Policy  that  establishes  that  curriculum  development  will  be  undertaken  only  when 
reimbursable  (FM;  I;  M) 

9.  Policy  that  better  exploits  technology  in  support  of  ELT,  with  clear  objective  of  reducing 
costs  wherever  possible  (FM;  T;  M) 

10.  Policy  that  clearly  establishes  that  DLIELC  is  ELT  organization  and  essential  BPC  instrument 
(I;  O/A) 

11.  Effective  assessment  process  (A) 

12.  Policy  to  expand  DLlELC's  labor  flexibility  (R;  FM;  I;  M) 

13.  Policy  that  seeks  to  improve  advocacy  for  DLIELC  by  senior  sponsors  (O/A) 


Priority 

Immediate 

Immediate 

Immediate 

Mid-term 

Mid-term 

Long-term 

Immediate 

Immediate 

Long-term 

Long-term 

Mid-term 

Long-term 

Mid-term 


NOTES:  M/P  =  Mission/Policy;  R  =  Requirements;  FM  =  Financial  Management;  T  =  Technology;  I  =  Identity; 

M  =  Manpower;  O/A  =  Organization  and  Advocacy.  Immediate  refers  to  execution  years.  Mid-term  refers  to  the 
FYDP.  Long-term  refers  to  FYDP  and  beyond. 
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CHAPTER  ONE 


Introduction 


Background 

The  Defense  Language  Institute  English  Language  Center  (DLIELC),  which  trains  U.S.  mili¬ 
tary  and  foreign  students  in  English,  plays  an  important  role  in  the  Department  of  Defense’s 
(DoD’s)  security  cooperation  objectives  as  a  key  enabler  and  strategic  engagement  tool.  Eng¬ 
lish  language  training  is  usually  required  prior  to  foreign  military  entry  into  DoD  and  Service 
training  and  professional  military  education  (PME)  courses. 

Despite  its  important  role,  DLIEC’s  mission  and  function  have  not  been  well  understood 
in  the  Air  Force.  More  specifically,  although  the  Air  Force  is  the  executive  agent  for  DLIELC, 
it  needs  to  have  a  better  capability  to  measure  DLIELC’s  cost  structure  and  the  mid-  and  long¬ 
term  demand  for  English  language  training  (ELT)  across  the  DoD  and  foreign  partners.  Resi¬ 
dent  and  nonresident  demand  for  ELT  exceeds  DLIELC’s  current  staffing  capacity,  and  there 
is  no  defined  mechanism  for  increasing  capacity  or  prioritizing  demand.  Without  a  capability 
to  measure  DLIEC’s  cost  structure  and  demand,  the  U.S.  Air  Force  is  unable  to  determine 
the  most  efficient  and  effective  training  models  to  optimize  foundational  and  surge  training.1 


DLIELC  Description 

Very  simply,  DLIELC  primarily  trains  foreign  students  in  the  English  language  as  a  precursor 
to  attendance  at  a  U.S.  military  school.  This  is  not  a  complete  definition  because  DLIELC  also 
trains  U.S.  citizens  enlisting  in  the  U.S.  Army.  Also,  not  all  foreign  students  go  on  to  further 
training  in  U.S.  military  schools.  Some  receive  only  English  language  instruction,  and  some 
are  trained  to  be  English  language  instructors. 

Approximately  2,400  students  are  trained  each  year,  with  an  average  peak  load  of  1,100 
and  a  minimum  of  500.  At  any  one  time,  there  can  be  90  or  more  countries  represented, 
although  in  recent  years,  the  majority  of  students  have  been  from  Saudi  Arabia. 

Demand  is  primarily  driven  by  weapon  purchases  and  the  resulting  need  to  train  for¬ 
eign  military  personnel  in  how  to  use  the  weapon  systems.  DLIELC  resides  in  San  Antonio, 
Texas,  on  Lackland  AFB.  It  has  been  in  existence  for  57  years.  It  has  resident  and  nonresident 
portions. 


1  While  the  Air  Education  and  Training  Command  (AETC)  has  supported  manpower  increases,  it  did  not  support 
DLIELC  military  construction  (MILCON)  request  and  could  not  state  whether  the  request  was  justifiable. 
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Objectives 

In  September  2010,  the  Air  Force  Language,  Region  and  Culture  Program  Office,  Policy  (AF/ 
A1DG),  and  the  Office  of  the  Secretary  of  Defense  for  Personnel  and  Readiness,  Defense  Lan¬ 
guage  Office  (OSD(P&R)/(DLO),  asked  RAND  Project  AIR  FORCE  to  examine  DLIELC  to 
ensure  that  it  is  appropriately  tasked,  organized,  operated,  resourced,  and  managed  to  produce 
the  ELT  capability  needed  to  effectively  and  efficiently  meet  DoD  mission  requirements.  The 
primary  objectives  of  the  study  were  completed  in  four  months  and  briefed  in  January  2011.  A 
cost  model  (Appendix  B)  was  developed  over  the  next  four  months. 

To  accomplish  those  objectives,  the  study  included  four  broad  tasks: 

1.  Review  prior  studies,  authorities,  and  governing  documents  for  DLIELC. 

2.  Develop  a  requirements  process  to  identify  and  predict  all  the  sources  of  student 
throughput  in  sufficient  detail  to  program  resource  needs. 

3.  Consider  options  to  improve  organization,  reporting,  and  resourcing  structure. 

4.  Assist  DLIELC  in  its  strategic  planning. 


Study  Approach 

Data  used  to  support  this  study  were  provided  primarily  by  DLIELC.  Our  study  approach  was 
examined  by  RAND’s  Fluman  Subjects  Protection  Committee  to  ensure  that  human  subjects 
were  properly  protected. 

The  study  team  used  a  rigorous  approach  to  collecting  and  analyzing  data  relative  to 
DLIELC.  To  gather  data  from  DLIELC,  we  first  used  a  written  question-and-answer  approach 
in  which  the  team  submitted  a  question  to  the  designated  DLIELC  official,  the  question  was 
then  researched,  and  the  answer  and  accompanying  data  were  provided  to  the  study  team  in  a 
timely  manner.  DLIELC  officials  answered  150  separate  requests  for  data;  in  only  a  few  cases 
were  data  not  provided.  In  this  document,  we  identify  those  cases  where  we  were  unable  to 
obtain  the  necessary  data  to  make  informed  judgments  and  factually  based  recommendations. 

As  part  of  our  data  collection,  we  reviewed  pertinent  regulations,  directives,  instructions, 
and  policy  documents  for  Joint,  Interservice,  and  Air  Lorce,  as  well  as  previous  studies  (Ravens 
Group,  2008;  Smith,  2004;  Duhon  and  Doyle,  2010)  and  RAND  security  cooperation-related 
work  (see  Appendix  A). 

Second,  we  conducted  in-person  interviews  on  several  occasions  through  on-site  visits 
with  officials  from  DLIELC,  including  the  AETC  Air  and  Space  Operations  Directorate 
Requirements  Division  (AETC/A3R),  International  Affairs  office  (AETC/IA),  and  the  Air 
Lorce  Security  Assistance  Training  Squadron  (ALSAT).  We  also  spoke  extensively  with  policy 
officials  in  Washington,  D.C.,  from  the  Defense  Language  Office  and  the  Defense  Security 
Cooperation  Agency  (DSCA).  The  primary  purposes  of  the  interviews  at  this  stage  were  to 
gain  official  understanding  of  processes,  to  uncover  potential  issues  to  investigate,  and  to  dis¬ 
cover  data  sources  for  validation. 

Third,  by  way  of  telephone  and  in-person  interviews  in  several  cases,  we  conducted  a 
comparative  analysis  with  other,  similar  organizations  that  have  a  security  cooperation  focus. 
We  analyzed  ten  DoD  organizations  and  two  foreign  English/Lrench-language  training  orga- 
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nizations  relative  to  DLIELC’s  eight  issue  areas.  The  ten  DoD  organizations  and  two  foreign 
organizations  were  the  following: 

•  Center  for  Civil-Military  Relations,  Naval  Postgraduate  School  (Monterey,  California) 

•  Center  of  Excellence  for  Disaster  Management  and  Elumanitarian  Assistance  (Honolulu, 
Hawaii) 

•  Defense  Institute  for  International  Legal  Studies  (Newport,  Rhode  Island) 

•  Defense  Institute  for  Medical  Operations  (San  Antonio,  Texas) 

•  Defense  Language  Institute  Foreign  Language  Center  (Monterey,  California) 

•  Defense  Institute  for  Security  Assistance  Management  (Dayton,  Ohio) 

•  Inter-American  Air  Forces  Academy  (San  Antonio,  Texas) 

•  DoD  Global  Center  for  Security  Cooperation  (Monterey,  California) 

•  Partner  Language  Training  Center  Europe  (Garmisch-Partenkirchen,  Germany) 

•  Western  Hemisphere  Institute  for  Security  Cooperation  (Ft.  Benning,  Georgia) 

•  Canadian  Forces  English/French  language  training  program 

•  United  Kingdom  Defence  School  of  Languages/English  Language  Wing  training 
program. 

The  comparative  analysis  helped  the  study  team  clarify  the  extent  to  which  DLIELC’s 
challenges  are  unique,  identify  best  practices  and  lessons  that  could  apply  to  DLIELC,  and 
identify  suitable  options  for  implementation. 


Organization  of  the  Report 

Chapters  Two  through  Nine  address  key  findings  identified  in  eight  critical  areas  identified  by 
the  study:  mission  and  policy,  requirements,  financial  management  and  cost  recovery,  technol¬ 
ogy,  identity,  assessment,  manpower,  and  organization  and  advocacy.  In  each  chapter,  we  first 
highlight  the  key  findings  and  then  develop  the  underlying  support  for  those  findings  in  the 
remainder  of  the  chapter.  We  close  each  chapter  with  a  set  of  recommendations  to  address  the 
issues  raised  in  the  findings. 

Chapter  Ten  presents  a  summary  of  the  findings,  key  recommendations,  and  a  proposed 
implementation  plan  to  address  the  issues  in  an  integrated  fashion.  Appendix  C  presents  the 
implementation  plan  in  more  detail.  Appendix  A  addresses  findings  on  security  partnership 
building  from  other  RAND  research,  and  Appendix  B  lays  out  a  proposed  cost  model. 


CHAPTER  TWO 


Mission  and  Policy 


This  chapter  provides  a  description  of  DLIELC,  beginning  with  a  brief  overview  of  the  orga¬ 
nization’s  mission  and  policy.  It  is  divided  into  two  sections.  The  first  section  provides  a  brief 
context  for  the  discussion  about  DLIELC  as  an  institution  within  the  broader  community  of 
security  cooperation  programs;  the  second  section  offers  an  overview  of  how  applicable  rules 
and  regulations  govern  the  milieu  in  which  DLIELC  operates. 

Based  on  our  analysis,  we  find  that  DLIELC  is  governed  by  a  maze  of  guidance.  More¬ 
over,  when  viewed  in  its  totality,  this  collection  of  guidance  does  not  accurately  reflect  the  real¬ 
ity  of  the  circumstances  within  which  DLIELC  must  operate.  These  findings  are  developed  in 
the  discussion  that  follows,  and  some  recommendations  to  address  these  concerns  are  included 
at  the  end  of  the  chapter. 


DLlELC's  Security  Cooperation  Mission 

DLIELC’s  official  vision  statement  portrays  the  organization  as:  “A  world-class  language  insti¬ 
tute  .  .  .  Building  bridges  through  communication  and  peace  through  understanding  .  .  .  Shap¬ 
ing  tomorrow’s  future  today.”  DLIELC’s  more  specific  mission  statement  describes  DLIELC 
as  the  “DoD  agency  responsible  for  the  management  and  operation  of  the  Defense  English 
Language  Program  (DELP)  to  train  international  military  and  civilian  personnel  to  speak  and 
teach  English,  manage  the  English  as  a  second  language  program  for  the  U.S.  military,  manage 
nonresident  English  training  programs,  provide  for  our  students’  health,  morale  and  welfare, 
and  conduct  the  DoD  Lield  Studies  Program.” 

As  the  mission  statement  suggests,  DLIELC  has  a  special  role  from  a  domestic  and  for¬ 
eign  perspective.  It  exists  first  and  foremost  to  teach  English  to  foreign  military  personnel  and 
to  train  English  language  instructors,  as  well  as  to  teach  English  to  U.S.  military  personnel  for 
whom  English  is  a  second  language.  DLIELC’s  mission  statement  specifies  the  tasks  performed 
by  the  organization  and  is  fairly  simple  and  uninspiring.  It  could  be  changed  to  more  sharply 
and  distinctly  connect  DLIELC’s  role  and  purpose  to  the  broader  security  cooperation  com¬ 
munity.  A  suggested  statement  follows: 

DLIELC  is  the  DoD  agency  responsible  for  the  management  and  operation  of  the  Defense 
English  Language  Program  to  build  enduring  partnerships  with  international  military  and 
civilian  personnel  and  prepare  them  for  follow-on  military  training.  Its  responsibilities 
include  managing  the  English  as  a  second  language  program  for  the  U.S.  military;  man¬ 
aging  nonresident  English  training  programs;  providing  for  students’  health,  morale  and 
welfare;  and  conducting  the  DoD  Field  Studies  Program. 
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As  far  as  DLIELC’s  place  within  the  broader  security  cooperation  community  is  con¬ 
cerned,  DLIELC  may  be  viewed  as  an  enabler  to  the  security  cooperation  mission:  For  many 
countries  around  the  world,  it  is  a  necessary  first  step  for  individuals  to  avail  themselves  of 
more  technical  training  provided  through  the  U.S.  security  assistance  process.  However,  the 
importance  of  DLIELC’s  role  could  be  stated  in  even  stronger  terms.  Some  DoD  officials  we 
spoke  with  described  the  organization  as  the  lynchpin  that  enables  the  United  States  to  build 
partner  capacity,  especially  in  the  realm  of  education  and  training,  in  many  countries  of  the 
world.  In  other  words,  education  and  advanced  technical  training  for  non-English  speakers  is 
simply  not  possible  until  they  first  acquire  the  appropriate  English  language  skills.  Without  the 
critical  role  played  by  DLIELC,  the  entire  process  could  fail. 


DLIELC  Rules  and  Regulations 

Beginning  with  a  series  of  DoD  directives  and  instructions,  DLIELC  is  governed  by  a  collec¬ 
tion  of  rules  and  regulations  covering  everything  from  program  administration  to  the  develop¬ 
ment  of  training  materials  (see  Table  2.1).  The  foundational  documents  that  govern  DLIELC 
begin  with  the  establishment  of  its  executive  agency  (the  Department  of  the  Air  Force)  and  go 
on  to  set  the  primary  responsibilities  for  its  oversight.  Like  most  DoD  organizations,  this  over¬ 
sight  responsibility  is  distributed  among  several  higher-level  organizations.  The  overarching 
guidance  document  is  Department  of  Defense  Directive  (DoDD)  5160.41E,  Defense  Language 
Program  (DLP),  in  which  the  Deputy  Secretary  of  Defense  assigns  responsibility  for  overall 
DLP  policy  guidance  to  the  Under  Secretary  of  Defense  for  Personnel  and  Readiness  and 
designates  the  Secretary  of  the  Air  Force  as  the  DoD  executive  agent  for  DLIELC.  From  this 
document,  headquarters  Air  Force  and  the  other  services  published  guidance  contained  in  Air 
Force  Instruction  (AFI)  16-105,  Joint  Security  Assistance  Training,  and  Air  Force  Joint  Instruc¬ 
tion  (AFJI)  16-103,  Managing  the  Defense  English  Language  Program. 

In  some  cases,  guidance  is  outdated  or  unworkable,  a  fact  highlighted  by  the  revisions 
under  way  in  AFI  16-105 ,  Joint  Security  Cooperation  Education  and  Training,  and  AFJI  16-103, 
Managing  the  Defense  English  Language  Program  (DELP).  These  revisions  will  address  critical 
areas  of  concern,  such  as  institutional  advocacy  and  organizational  structure. 

In  other  cases,  DLIELC  is  exempt  from  following  the  rules  that  other  similar  training 
organizations  must  follow.  For  example,  despite  the  fact  that  DLIELC  is  a  part  of  AETC’s 
technical  training  structure,  Second  Air  Force  (2AF)  exempts  DLIELC  from  complying  with 
certain  instructions  about  designing  and  developing  training  systems  or  evaluating  training 
programs.1  Within  the  community  of  AETC  training  organizations,  this  places  Headquarters 
(HQ)  AETC  managers  at  a  disadvantage,  both  in  terms  of  being  able  to  clearly  understand 
DLIELC  and  its  effectiveness  and  by  removing  access  to  key  AETC  management  processes. 

Finally,  in  some  cases,  existing,  applicable  rules  and  regulations  are  not  fully  complied 
with,  or  provisions  are  not  fully  exercised.  For  example,  the  lack  of  a  course  costing  model  and 
the  difficulty  in  finding  cost  data  suggest  that  the  rules  for  reimbursement  are  not  followed. 


1  2AF  is  the  intermediate  headquarters,  subordinate  to  HQ  AETC,  responsible  for  managing  the  Air  Force’s  technical 
training.  The  37th  Training  Wing,  to  which  DLIELC  reports,  is  directly  subordinate  to  2AF. 
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Table  2.1 

Summary  of  Guidance 


Level 

Regulation 

Title 

OSD 

DoDD  1400.5 

DoDD  1400.25 

DoD  1400. 25-M 

DoDD  5101.1 

DoDD  5160.41E 

DoDI  5160.70 

DoDI  5160.71 

DoDM  5105. 38-M  Chapter  10 

DoDM  5105. 38-M  Chapter  14 

DoDM  5105. 38-M  Chapter  5 

DoD  Policy  for  Civilian  Personnel 

Civilian  Personnel  Management  System 

Civilian  Personnel  Manual 

DoD  Executive  Agent 

Defense  Language  Program  (DLP) 

Management  of  DoD  Language  and  Regional  Proficiency 
Capabilities 

DoD  Language  Testing  Program 

Security  Assistance  Management  Manual — International 
Training 

Security  Assistance  Management  Manual — Forecasting, 
Programming,  Budgeting,  and  Audits 

Security  Assistance  Management  Manual — FMS  Case 
Development 

Joint/ 

Interservice 

AFJI  16-103  (under  review) 

AFI  16-105 

AFI  16-105  (in  draft) 

AFI  36-2230(1) 

Managing  the  Defense  English  Language  Program  (DELP) 

Joint  Security  Assistance  Training 

Joint  Security  Cooperation  Education  and  Training 

Inter-Service  Training 

Air  Force 

AFPD  36-2 

AFPD  36-22 

AFI  16-501 

AFI  36-2201,  Vol.  1 

AFI  36-2616 

AFMAN  16-101 

AFMAN  36-203 

AFMAN  36-2234 

AFH  36-2235,  Vol.  4 

Civilian  Recruitment  and  Placement 

Air  Force  Military  Training 

Control  and  Documentation  of  Air  Force  Programs  (PPBE) 

Training  Development,  Delivery,  and  Evaluation 

Trained  Personnel  Requirements 

International  Affairs  and  Security  Assistance  Management 

Staffing  Civilian  Positions 

Instructional  System  Development 

Information  for  Designers  of  Instructional  Systems:  Manager's 
Guide  to  New  Education  and  Training  Technologies 

A  ETC 

AETCI  36-2201 

AETCI  36-2203 

AETCI  36-2215 

Technical  and  Basic  Military  Training  Evaluation 

Technical  and  Basic  Military  Training  Development 

Technical  and  Basic  Military  Training  Administration 

DLIELC 

DLIELC  Instruction  1025-7 

DLIELC  Instruction  1025.15 

DLIELC  Instruction  1025.30 

DLIELC  Instruction  1025.9 

Planning  and  Programming  Security  Assistance  English 

Language  Training 

English  Comprehension  Level  (ECL)  Test  Guidelines 

English  Language  Training  for  International  Military  Students 

Management  of  the  DLIELC  Oral  Proficiency  Interview  (OPI) 
Program 

Requirements  Identification 

AFI  36-2230,  Interservice  Training,  is  a  joint  regulation  (i.e.,  it  is  used  by  all  Services,  with  each 
Service  assigning  to  the  document  its  own  unique  designator);  this  places  the  responsibility  for 
requirements  identification  on  the  host  Service.  For  DLIELC,  this  means  that  the  Air  Force 
has  the  burden  for  identifying  requirements. 


8  Optimizing  the  Defense  Language  Institute  English  Language  Center 


Air  Force  Instruction  (AFI)  16-105,  Joint  Security  Assistance  Training,  provides  detailed 
guidance  about  the  oversight  and  day-to-day  management  of  ELT.  Because  the  training  is 
an  inter-Service  enterprise,  the  instruction  contains  guidance  for  all  Services.  Section  4  of 
the  instruction  provides  general  guidance  for  determining  security  assistance-related  training, 
including  training  conducted  to  support  international  military  education  and  training  (IMET) 
and  foreign  military  sales  (FMS)  needs.  In  general,  the  process  is  quite  formalized,  and  the 
documentation  accounts  for  the  chain  of  command,  beginning  with  the  in-country  security 
assistance  officer,  to  the  combatant  commands  (COCOMs),  and  on  through  to  Defense  Secu¬ 
rity  Cooperation  Agency  (DSCA)  and  the  military  departments.  The  process  includes  formal 
reviews  and  recurring  meetings  and  conferences  to  ensure  that  training  requirements  are  accu¬ 
rate  and  that  the  training  infrastructure  can  support  them. 

Chapter  3  of  AFI  16-105  describes  how  the  ELT  interfaces  with  the  training  require¬ 
ments  process.  Inherent  in  the  process  described  above  is  the  development  of  security  assistance 
cases,  beginning  with  a  letter  of  request  (LOR)  and  ending  with  a  letter  of  offer  and  accep¬ 
tance  (LOA).  Paragraph  3-3a  states  simply,  “those  Letters  of  Offer  and  Acceptance  (LOA)  that 
include  provisions  for  ELT  must  be  coordinated  with  DLIELC  before  negotiation.”  In  other 
words,  DLIELC  must  be  afforded  the  opportunity  to  review  all  LOAs  to  determine  if  it  has  the 
capacity  to  provide  the  requested  training.  This,  along  with  participation  in  the  routine  train¬ 
ing  requirements  processes,  is  intended  to  give  DLIELC  insight  into  the  totality  of  the  ELT 
requirements  at  any  given  time. 

Financial  Risk 

Two  areas  in  which  DLIELC  receives  guidance  about  financial  risk  are  found  in  AFI  16-105. 
In  Chapters  3  (requirements)  and  4  (financial  management),  the  issues  of  tuition  pricing  and 
reimbursement  (or  “forfeiture”)  for  missed  training,  or  cancellations,  are  described.  AFI  16-105 
refers  to  DoD  Financial  Management  Regulation  7000. 14-R,  Volume  15,  Chapter  7,  which  is 
entitled  “pricing.”  The  general  policy  for  determining  tuition  prices  for  the  training  of  foreign 
nationals  is  contained  in  section  710,  while  the  actual,  detailed  procedures  for  computing  the 
tuition  are  contained  in  sections  711  and  712. 

AFI  16-105  describes  in  detail  the  procedure  for  collecting  reimbursement  for  expenses 
incurred  for  a  variety  of  reasons.  Paragraph  3-7,  for  example,  allows  AETC  Financial  Manage¬ 
ment  to  assess  50  percent  of  the  tuition  for  late  cancellations,  reschedules,  or  no-shows.  Once 
a  student  has  entered  training,  DLIELC  can  also  charge  an  additional  amount  if  the  training 
takes  longer  than  expected.  In  the  event  of  a  late  arrival,  DLIELC  may  assess  50  percent  of  the 
tuition  for  the  number  of  weeks  late,  up  to  a  maximum  of  50  percent  of  the  scheduled  train¬ 
ing.  Finally,  DLIELC  may  charge  for  the  actual  number  of  weeks  completed,  but  not  less  than 
50  percent  of  the  training  time,  for  students  who  fail  to  pass  the  training. 

Technology 

The  Air  Force  enthusiastically  embraces  the  use  of  technology  in  support  of  technical  training. 
While  acknowledging  the  limitations  that  budgets,  physical  facilities,  and  unique  geographical 
or  other  circumstances  may  impose,  the  Air  Force  tells  its  training  managers  that: 

.  .  .  instructional  design  offers  great  potential  for  creativity  and  cost  avoidance  through 

application  of  state-of-the-art  instructional  technology  and  advances  in  management,  com¬ 
munication,  and  behavioral  science  (Air  Force  Flandbook  36-2235,  Volume  1,  2002,  p.  16). 
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Moreover,  the  Air  Force  routinely  exhorts  its  training  managers  to  entertain  new  and 
emerging  technologies,  evaluating  how  they  might  enhance  training  and  more  effectively  pro¬ 
duce  the  desired  outcomes.  Importantly,  the  Air  Force  reminds  its  commanders  and  training 
managers  to 

.  .  .  maintain  the  focus  on  mission  accomplishment.  Evaluate  the  technologies  in  terms  of 
the  education  and  training  of  Air  Force  students  (Air  Force  Flandbook  36-2235,  Volume 
4,  2002,  p.  14). 

Although  it  is  clear  that  DLIELC  values  the  use  of  technologies  in  its  training  delivery, 
the  study  team  considered  additional  ways  in  which  technology  could  increase  the  schooFs 
ability  to  respond  to  fluctuations  in  student  load  and  limitations  on  instructors  and  other 
resources.  (These  are  discussed  below  in  Chapter  Five.) 

Training  Assessment 

The  1994  version  of  AFJI  16-103  requires  DLIELC  to  evaluate  ELT  to 

Identify  strengths  and  weaknesses  in  the  [English  language  training  program,  or  ELTP], 
find  out  if  course  objectives  adequately  serve  follow-on  training  needs,  find  out  whether 
courses  meet  the  stated  objectives,  [and]  recommend  changes  that  will  improve  efficiency 
and  effectiveness  of  ELTP  (paragraph  1.7). 

Moreover,  it  tasks  agencies  to  notify  DLIELC  when  they  believe  that  ELT  training  was 
“substandard  or  unsatisfactory.”  However,  this  evaluation  process  is  not  completely  clear. 
AETC  Instruction  36-2201,  Technical  and  Basic  Military  Training  Evaluation,  establishes  pro¬ 
cedures  and  responsibilities  to  assess  the  quality  of  technical  training.  This  instruction  applies 
to  the  Inter-American  Air  Forces  Academy  (IAAFA)  and  the  technical  training  schools  at  Air 
University  and  under  2AF,  but  it  does  not  apply  to  DLIELC.  Instead,  all  inter-Service  training 
is  referred  to  AFI  36-2230,  Interservice  Training  AETC  Instruction  36-2201,  Training  Devel¬ 
opment,  Delivery,  and  Evaluation,  makes  this  clear,  saying  (2005,  p.  13): 

.  .  .  evaluate  interservice  courses  in  accordance  with  AFI  36-2230,  Interservice  Training, 
and  applicable  interservice  agreements.  Use  caution  to  ensure  AETC  does  not  duplicate 
existing  host-service  evaluations  that  are  adequate.  The  chief  of  the  training  evaluation 
unit  may  include  or  exclude  other  service  graduates  from  surveys  except  when  specifically 
requested  to  do  so  by  officials  of  other  services. 

AFI  36-2230,  which  generally  sets  policies  and  procedures  for  inter-Service  training  of 
military  and  civilian  personnel  of  the  DoD,  assigns  responsibility  for  evaluation  to  host  Ser¬ 
vices,  but  it  does  not  specify  how  evaluations  will  be  conducted.  Instead,  it  states  simply  that 
the  host  Service  will  “be  primarily  responsible  for  consolidated  course  revisions  and  evaluations 
(AFI  36-2230,  2006,  paragraph  6.b.(4)).”  Therefore,  DLIELC  is  on  its  own  to  determine  how 
best  to  accomplish  any  evaluation  of  the  program.  As  noted  above,  HQAETC’s  evaluation 
processes  (per  AETC  Instruction  36-2201)  do  not  apply  to  DLIELC,  and  HQ  Air  Force  (AF) 
passes  the  responsibility  directly  to  DLIELC,  per  AFJI  16-103,  saying  that  DLIELC  “assesses 
the  quality  of  ELT  activities.”  (This  language  is  contained  in  both  the  current  and  draft  version 
of  the  instruction.)  Unfortunately,  we  found  little  evidence  of  assessment. 
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Recommendations 

As  noted  at  the  start  of  this  chapter,  DLIELC  is  a  complex  organization  with  a  maze  of  guid¬ 
ance.  Additionally,  that  guidance  does  not  necessarily  reflect  the  reality  of  the  circumstances 
within  which  DLIELC  must  operate.  Its  mission  is  challenging,  and  the  conditions  under 
which  it  operates  can  seem  unpredictable. 

Given  this  finding,  we  recommend  that 

•  DLIELC  revise  its  mission  statement  to  sharply  and  distinctly  connect  DLIELC’s  impor¬ 
tant  role  in  the  security  cooperation  community.  A  suggested  statement  is  provided  above 

•  HQ  Air  Lorce  Language,  Region  and  Culture  Program  Office,  Policy  undertake  a  full 
review  of  all  applicable  rules  and  regulations  to  ensure  they  are  current,  appropriate,  and 
well-aligned.  Optimally,  there  would  be  two  instructions,  a  governing  DoD  instruction 
and  a  consolidated  AP  instruction  covering  everything  needed  to  grade,  audit,  govern, 
cost,  and  direct  the  organization2 

•  DLIELC  consider  leveraging  existing  AETC  processes  and  procedures  where  they  may 
create  opportunities  to  fully  tap  into  AETC’s  resources  and  expertise 

•  HQAP/A1DG  review  existing  rules  and  regulations  to  look  for  underutilized  authorities 
and  provisions  that  could  be  exercised  to  enhance  DLIELC’s  effectiveness  in  the  provi¬ 
sion  of  cost-efficient  and  content-effective  ELT. 


2  One  of  RAND’s  tasks  was  to  review  existing  regulations.  Although  we  compiled  a  list  of  regulations,  it  was  far  outside 
the  scope  of  that  task  to  create  the  two  governing  documents  that  ought  to  guide  DLIELC. 


CHAPTER  THREE 

Requirements 


Existing  guidance  points  to  the  need  to  fully  account  for  ELT  requirements  while  planning 
for  DoD  training. 

According  to  DoD  5105. 38-M,  Security  Assistance  Management  Manual,  Chapter  10, 
“International  Training,”  the  time  required  to  conduct  training  must  be  taken  into  consider¬ 
ation  when  estimating  delivery  dates  of  equipment.  Training  programs  must  consider  the  avail¬ 
ability  of  personnel;  the  skills  to  be  developed  (including  ELT);  and  the  time  required  to  plan, 
implement,  and  complete  the  program.  In  other  words,  ELT  needs  to  be  part  of  the  training 
planning  process  and  cannot  be  an  afterthought. 

In  this  chapter,  we  discuss  past  and  current  demand  trends,  estimating  demand,  man¬ 
aging  demand,  and  managing  supply.  To  fully  understand  the  problem,  the  study  team  con¬ 
ducted  a  rigorous  analysis  of  current  and  historical  trends  in  student  load  and  related  factors. 
Moreover,  the  team  used  a  variety  of  statistical  analysis  techniques  and  explored  a  number  of 
ways  to  more  fully  identify  and  respond  to  variations  in  requirements. 

Based  on  our  analysis,  we  find  that  DLIELC  lacks  a  robust  requirements  determination 
process  and  that  the  organization’s  business  model  is  unable  to  meet  variable  demand — either 
effectively  or  efficiently. 

Before  moving  on  to  the  remainder  of  this  chapter,  we  note  that  in  addition  to  resident 
training,  DLIELC  provides  nonresident  training  through  in-country  mobile  training  teams 
(MTTs)  and  language  training  detachments  (LTDs).  This  nonresident  activity  is  sizable.  In 
fiscal  year  (FY)  2010,  DLIELC’s  MTTs  trained  a  total  of  2,995  international  students  in  25 
countries.  DLIELC’s  LTDs  trained  1,580  students.1  Nonresident  training  data  prior  to  FY10 
were  unavailable;  therefore,  we  were  unable  to  analyze  the  requirement  process  and  past  trends 
for  MTT  and  LTD  activities.  As  a  result,  the  following  section  focuses  exclusively  on  resident 
training  requirements. 


Past  and  Current  Demand  Trends 

Overall,  DLIELC’s  total  student  load  increased  steadily  over  the  past  ten  years,  from  2,238 
students  starting  a  course  at  DLIELC  in  FY01  to  3,119  in  FY09  (Figure  3.1).  The  number  of 
international  military  students  (IMSs)  increased  in  equal  measure,  with  the  exception  of  a 


1  Nonresident  training  can  also  have  an  impact  on  the  number  of  students  attending  DLIELC  in  San  Antonio  because 
MTTs  occasionally  suggest  that  host  countries  send  some  of  their  students  to  DLIELC  for  specialized  English  training 
(SET)  (email  correspondence  with  DLIELC,  December  17,  2010). 


11 


12  Optimizing  the  Defense  Language  Institute  English  Language  Center 


Figure  3.1 

Number  of  Students  Starting  a  Course  at  DLIELC  by  Year,  FY01-FY09 


RAND  TR1 1S2-3. 1 


brief  drop  in  FY07.  The  number  of  U.S.  students  remained  in  the  530-620  range  in  FY01-07, 
increasing  to  800  after  that  date. 

Central  Command  (CENTCOM)  accounted  for  the  largest  part  of  this  increase,  with 
numbers  of  students  from  other  combatant  commands  (COCOMs)  remaining  stable  or  even 
decreasing  (in  the  case  of  European  Command  [EUCOM]),  as  shown  in  Figure  3.2. 


Figure  3.2 

IMS  Repartition  by  COCOM,  Percentage  of  Overall  IMS  Load,  FY06-FY09 
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The  large  influx  of  Saudi  students  explains  most  of  this  increase.  The  percentage  of  Saudi 
students  among  IMSs  almost  doubled  in  ten  years,  from  10.6  percent  of  all  IMSs  in  2000- 
2004  to  19  per  cent  in  2005-2010.  Their  increase  has  been  linear  since  FY05  (Figure  3.3). 

More  IMSs  have  been  attending  SET  only,  and  this  trend  is  even  more  salient  for  IMSs 
attending  generalized  English  training  (GET)  and  SET.2  Their  numbers  almost  doubled  over 
the  past  ten  years,  from  541  in  2000  to  1,076  in  2009.  The  number  of  IMSs  attending  GET 
only  also  increased  steadily  over  time,  but  much  less  so.  In  2009,  only  169  IMSs  (7  percent  of 
the  total  number)  attended  GET  only. 

U.S.  Service  Demand 

All  Services  are  represented  at  DLIELC,  but  Army-sponsored  students  dominate  the  overall 
student  load.  The  Air  Force  is  still  well  represented  among  IMSs:  Air  Force-sponsored  IMSs 
accounted  for  32  to  51  percent  of  DLIELC’s  international  customers  in  FY01-07.3  They  were 
on  a  par  with  Army-sponsored  IMSs  (34  to  50  percent  of  the  total  IMS  load).  In  more  recent 
years,  however,  the  share  of  Army-sponsored  students  among  IMET  and  FMS-funded  IMSs 
increased  steadily,  while  the  share  of  Air  Force-sponsored  students  decreased  slightly  (38  per¬ 
cent  in  2007,  37  percent  in  2008,  35  percent  in  2009).  With  the  addition  of  ECHO  Company, 
Army-sponsored  students  therefore  represent  a  clear  majority  among  DLIELC  customers.4 


Figure  3.3 

Percentage  of  Saudi  Students  Among  IMSs,  FY01-FY09 
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2  GET  teaches  the  basics  of  the  English  language  using  a  common  vocabulary.  SET  assumes  a  working  knowledge  of 
English  and  teaches  a  vocabulary  specific  to  a  vocational  or  professional  occupation. 

3  Calculations  could  be  made  only  for  FY07  and  prior  years,  because  the  Navy  and  Army  started  using  “D”  (Air  Force) 
military  article  service  listings  (MASLs)  after  FY08.  The  reason  for  this  change  is  their  transitioning  to  the  Defense  Security 
Assistance  Management  System  program,  which  only  uses  “D”  MASLs. 

4  ECHO  Company  is  a  U.S.  Army-only  company  of  recruits,  deficient  in  English,  who  learn  English  prior  to  attendance 
at  Army  basic  training. 
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Overall,  DLIELC  has  limited  control  and  visibility  over  requirements,  which  are  largely 
determined  by  other  agencies — military  departments,  security  cooperation  officers  (SCOs), 
or  follow-on  training  (FOT)  sites.  FMS  cases  largely  depend  on  foreign  governments’  defense 
budgets,  which  can  be  quite  unpredictable  (including  in  cases  where  a  country’s  wealth  depends 
largely  on  the  price  of  oil  and  other  commodities,  which  are  susceptible  to  sudden  changes). 
Examination  of  past  trends  at  DEIEEC  shows  that  in  2007-2009,  the  number  of  Navy- 
sponsored,  FMS-funded  students  remained  stable — around  160  students  per  year — while  the 
number  of  Air  Force-sponsored  and,  to  an  even  larger  extent,  Army-sponsored  FMS  students 
increased.5 

The  number  of  IMET-funded  students  has  proven  more  stable  (around  800  per  year 
in  2007-2009),  but  the  National  Security  Strategy’s  increased  focus  on  building  partnership 
capacity  (BPC)  could  lead  to  an  IMET  increase  in  the  near  future. 

Seasonal  Peaks 

Requirements  vary  considerably  throughout  the  year,  with  DEIEEC  experiencing  a  sharp  peak 
in  student  load  in  May  and  June.  This  peak  stretches  DLIELC’s  resources  (both  staff  and  infra¬ 
structure)  to  an  uncomfortable  point.  In  FY09,  this  peak  brought  an  additional  502  students 
above  the  anticipated  low  peak  demand.  The  demand  range  in  FY10  was  smaller,  bringing 
only  an  additional  370  students.  The  size  of  the  demand  range  is  primarily  caused  by  multiple 
FOT  sites  beginning  their  training  at  the  same  time  (in  late  summer  and  early  fall),  resulting  in 
large  numbers  of  students  attending  DLIELC  simultaneously.  Over  the  past  two  years,  the  size 
of  the  demand  range  has  decreased,  but  DLIELC  still  had  a  60  percent  student  load  increase 
during  the  FY10  peak.  Forecasting  peak  requirements  is  difficult:  Seasonal  peaks  increased 
from  FY07  to  FY09  but  decreased  in  FY10  (Figure  3.4).  Demand  range6  has  decreased  since 
FY08.  The  English  Language  Center  Requirements  Process  Panel  examined  the  possibility  of 
changing  some  FOT  start  dates,  but  this  solution  proved  impractical.7 


Estimating  Demand 

In  theory,  processes  to  view  the  training  pipeline  in  its  entirety,  including  ELT  requirements, 
exist.  Every  year  in  June,  DSCA  sends  a  tasking  message  on  the  Javits  Report8  and  two  years 
(budget  and  plan  fiscal  years)  of  FMS  forecasting  projections  to  implementing  agencies  and 
SCO.  In  August,  SCO  sends  its  input  to  implementing  agencies  and  DSCA;  implementing 
agencies,  in  turn,  provide  their  input  to  DSCA.  In  early  December,  implementing  agencies 
discuss  their  issues  or  concerns  with  the  upcoming  programming  process  during  a  program¬ 
ming  process  meeting.  Issues  raised  include  both  programming  cycle  processes  and  priorities 


5  These  observations  are  based  on  three  data  points  only  (2007,  2008,  and  2009)  and  should  therefore  be  treated  with  cau¬ 
tion  and  tested  against  future  data. 

6  Demand  range  is  the  highest  number  of  students  versus  the  lowest  number  of  students. 

7  Focused  discussions  with  AF/A1DG,  October  22,  2010. 

8  In  accordance  with  paragraph  25  of  the  Arms  Export  Control  Act,  the  State  Department  is  required  to  provide  to  Con¬ 
gress  an  Arms  Sale  Proposal  (the  “Javits  Report”)  covering  all  sales  and  licensed  commercial  exports  of  major  weapons  or 
weapon-related  defense  equipment  for  $7,000,000  or  more,  or  of  any  other  weapons  or  weapons-related  defense  equipment 
for  $25,000,000  or  more,  which  are  considered  eligible  for  approval. 
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Figure  3.4 

Demand  Range,  FY07-FY10 


FY07  FY08  FY09  FY10 


for  possible  inclusion  in  the  DSCA  Programming  Guidance.  DLIELC  receives  digital  notifica¬ 
tion  of  training  through  the  Security  Assistance  Network  web-based  database  and  may  receive 
notice  from  the  Air  Force,  COCOM,  or  DoD  agency  representatives  involved  in  FMS,  IMET, 
security  force  assistance,  and  other  training  programs. 

In  reality,  DEIEEC  is  placed  in  a  dilemma:  Although  it  is  a  provider  of  a  service,  it  finds 
that  it  is  not  set  up  to  be  a  seller  of  that  service,  thereby  undermining  the  case  for  retaining  its 
current  business  model.  It  is  largely  unable  to  forecast  accurately  how  many  students  will  be 
attending  the  school  at  a  given  time  because  of  several  factors.  International  travel  orders  state 
only  “Report  on  or  about  [date],”  and  security  assistance  network/student  training  list  projec¬ 
tions  may  only  indicate  the  quarter  in  which  a  student  is  due  to  arrive.9  In  some  instances,  there 
is  no  advance  notification.  IMSs  can  show  up  early  or  late  or  can  be  replaced  at  the  last  minute 
by  another  candidate.  DEIEEC  does  not,  as  a  rule,  turn  IMSs  away  and  has  to  accommodate 
them  whether  it  received  prior  notice  or  not.10  This  has  resulted  in  the  student  load  at  DLIELC 
routinely  exceeding  the  school’s  maximum  capacity  of  870  seats.  Late  arrivals  (because  of  a 
variety  of  reasons,  ranging  from  visa  delays  to  personal  or  dependent-related  issues)  complicate 
the  planning  and  scheduling  process.  The  percentage  of  late  arrivals  has  increased  in  frequency, 
gaining  2  to  3  percentage  points  each  year  since  FY06  (Figure  3.5),  and  the  delays  have  also 
increased  an  additional  0.2  to  0.7  weeks  per  year  since  FY06. 


9  Email  from  DLIELC,  October  6,  2010. 

10  According  to  a  DLIELC  source,  “Typically  we  make  them  stay  off  base  till  military  department/SCO  get  their  docs 
right,”  but  they  are  not  turned  away  (DLIELC  email,  October  6,  2010). 
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Figure  3.5 

Percentage  of  IMSs  Who  Start  Their  Training  Late,  FY05-FY09 
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DLIELC’s  ability  to  obtain  additional  resources  to  meet  increasing  requirements  has 
been  hampered  by  its  lack  of  a  reliable  demand  forecasting  process.11  An  English  Language 
Center  Requirements  Process  Panel  was  set  up  in  2010  to  address  this  issue  and  develop  a  solid 
requirements  projection  process  on  which  to  base  future  requests.  As  a  first  step,  the  panel 
asked  military  departments  to  provide  their  FMS  projections  for  (1)  six  months  to  one  fiscal 
year;  (2)  one  to  two  fiscal  years;  and  (3)  three  to  four  fiscal  years.  This  request  to  Services  will 
be  repeated  every  quarter,  with  the  panel  becoming  permanent. 

Ideally,  DLIELC  would  know  requirements  two  to  five  years  ahead  to  have  sufficient  time 
to  plan  for  new  buildings  and  major  infrastructure  changes.  Knowing  requirements  one  to  two 
years  in  advance  is  the  minimum12  for  the  school  to  be  able  to  contract  out,  train  instructors, 
etc.  It  may  need  less  time  if  a  robust  contracting  vehicle  were  already  in  place.13  The  panel  is 
currently  focusing  its  work  on  improving  DLIELC’s  flexibility  on  the  supply  side. 

Demand  Forecasting  Models 

Planning  and  forecasting  student  enrollment  is  a  fundamental  task  for  training  institutions 
and  technical  schools  throughout  DoD.  Even  though  the  topic  has  been  thoroughly  researched 
over  the  past  few  decades,  consensus  has  not  been  reached  on  an  optimal  model  to  accurately 
forecast  enrollment.  Although  there  is  no  single  best  approach,  research  suggests  identifying 


11  DSCA’s  projection,  for  the  FY12  Program  Objective  Memorandum  (POM),  of  1,400  FMS  cases  by  FY14  called  for  an 
additional  $38  million  in  resources  for  the  school,  but  this  request  was  rejected  by  AETC  during  the  POM  process  for  lack 
of  substantiating  evidence  of  an  increase  in  requirements. 

12  A  proven  and  robust  indefinite  delivery/indefinite  quantity  (IDIQ)  contracting  vehicle  can  respond  to  demand  increases 
in  months  versus  the  six  months  to  a  year  to  hire  new  instructors. 

13  Source  for  the  entire  paragraph  except  IMET  figures  and  evolution  of  FMS  cases  by  services  focused  discussion  with  AF/ 
A1DG,  October  22,  2010. 
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both  manageable  and  unmanageable  factors.  Unmanageable  factors  beyond  the  control  of  an 
institution  may  include  competition,  public  policy,  and  economic  factors.  In  contrast,  control¬ 
lable  and  manageable  factors  might  include  admissions  policies,  marketing  efforts,  and  dis¬ 
missal  policies.  The  most  notable  finding  is  that  DLIELC  has  few,  if  any,  manageable  factors 
to  control  enrollment. 

Therefore,  the  variability  in  enrollment  for  ELT  at  DLIELC  is  driven  primarily  by  one 
critical  unmanageable  factor:  FMS.  FMS  is  a  challenge  for  DLIELC  because,  unlike  IMET, 
for  example,  FMS  cases  are  episodic  and  not  predictable.  For  instance,  the  opening  of  an  FMS 
case  does  not  necessarily  guarantee  that  students  will  be  attending  DLIELC.  Nevertheless, 
estimates  of  FMS  provided  in  Congressional  Budget  Justification  (CBJ)  reports  for  each  coun¬ 
try  may  serve  as  indicators  or  predictors  of  student  attendance. 

For  any  given  year,  the  CBJ  provides  one-year-out  and  two-year-out  estimates  for  FMS. 
For  example,  for  FY11,  two  separate  FMS  estimates  were  provided  for  Afghanistan:  A  two- 
year-out  estimate  in  the  FY09  CBJ  and  a  one-year-out  estimate  in  the  FY10  CBJ.  Although 
FMS  estimates  could  predict  student  enrollment  for  each  country,  we  developed  a  model  using 
aggregate  FMS  estimates  for  each  COCOM  to  provide  a  more  robust  estimate  of  anticipated 
total  student  load. 

Because  some  countries  have  never  sent  students  to  DLIELC,  only  a  subset  of  estimates 
was  used  to  develop  the  COCOM  estimates.  Countries  with  regular  FMS  transactions  that 
have  never  sent  students  to  DLIELC  are  often  English-speaking  nations.  Other  countries  are 
inconsistent  or  low-demand  customers.  Therefore,  we  computed  aggregate  estimates  using  only 
those  countries  that  had  sent  at  least  20  students  to  DLIELC  over  the  previous  nine  years. 
These  countries  sent  a  total  of  301  students  to  DLIELC  over  this  nine-year  period,  resulting 
in  an  average  of  1.5  students  for  each  country  each  year.  To  account  for  the  demand  of  these 
countries,  an  additional  constant  of  34  students  should  be  added  to  the  student  load  projection 
described  below. 

FMS  Enrollment  Lag 

Student  enrollment  at  DLIELC  does  not  necessarily  occur  during  the  same  year  as  a  com¬ 
pleted  FMS  transaction.  There  is  usually  some  unspecified  time  lag  that  occurs  between  the 
completed  FMS  transaction  and  student  enrollment  at  DLIELC.  Therefore,  student  enroll¬ 
ment  may  be  predicted  by  CBJ  estimates  for  two  and  three  years  prior  to  the  year  of  student 
enrollment. 

Based  on  these  time-lag  assumptions,  we  examined  FMS  estimates  for  two  and  three 
years  prior  to  the  target  year  of  projected  enrollment.  For  example,  to  predict  student  enroll¬ 
ment  for  FY10,  we  used  one-  and  two-year-out  FMS  estimates  for  FY07  and  FY08.  As  men¬ 
tioned  previously,  these  estimates  were  then  aggregated  for  each  COCOM.  Bivariate  correla¬ 
tions  indicated  strong  positive  relationships  between  COCOM  FMS  estimates  and  COCOM 
student  load  (Table  3.1). 

The  study  team  conducted  a  stepwise  regression  analysis  to  identify  which  set  of  FMS 
estimates  explains  the  most  variance  in  student  load.  The  results  of  this  analysis  indicated  that, 
except  for  the  two-year-out  estimate  for  three  years  prior  to  enrollment,  all  estimates  contrib¬ 
uted  incrementally  to  the  prediction  of  student  load.  Based  on  these  results,  a  composite  FMS 
estimate  was  computed  using  the  following  estimates,  equally  weighted: 
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Table  3.1 

Descriptive  Statistics  and  Correlations  Between  FMS  Estimates  and  Student  Load 


Mean 

SD 

1 

2 

3 

4 

5 

1 

One-year-out  estimate  (two  years  prior 
to  enrollment 

1.94 

2.23 

1 

2 

Two-year-out  estimate  (two  years  prior 
to  enrollment) 

1.69 

2.08 

.75 

1 

3 

One-year-out  estimate  (three  years 
prior  to  enrollment) 

1.75 

1.90 

.83 

.76 

1 

4 

Two-year-out  estimate  (three  years 
prior  to  enrollment) 

1.69 

2.02 

.68 

.71 

.80 

1 

5 

Student  load 

339 

298 

.89 

.87 

.88 

.76 

1 

NOTES:  SD  =  standard  deviation.  Means  and  SDs  are  scaled  to  $1  billion.  The  means,  SDs,  and  correlations 
were  computed  using  estimates  and  student  load  aggregated  for  each  COCOM.  There  are  35  data  points. 
The  data  were  from  2004-2009  for  all  COCOMs.  Data  were  not  available  for  Northern  Command  in  2007. 


1.  two-year-out  estimate  (two  years  prior  to  enrollment  using  one-year-out  FMS  estimate) 

2.  two-year-out  estimate  (two  years  prior  to  enrollment  using  two-year-out  FMS  estimate) 

3.  three-year-out  estimate  (three  years  prior  to  enrollment  using  one-year-out  FMS 
estimate). 

Linear  regression  was  used  to  test  how  well  the  COCOM  FMS  composite  estimates 
predicted  student  load  at  DLIELC.  The  results  indicated  that  91  percent  of  the  variance  in 
DLIELC  student  load  was  accounted  for  and  that  student  load  can  be  predicted  up  to  two 
years  in  advance  (see  Table  3.2). 

The  first  step  in  projecting  student  load  requires  aggregating  FMS  estimates  (i.e.,  see 
list  items  1-3  above)  for  each  COCOM  and  dividing  by  3,  to  provide  an  average.  The  final 
estimate  for  total  student  load  should  also  account  for  the  average  34  students  each  year  from 
countries  that  were  not  included  in  the  COCOM  FMS  estimates.  More  specifically,  34  should 
be  added  to  the  sum  of  the  COCOM  estimates  derived  from  the  following  formula: 

(71.22  +  COCOM  FMS  Estimate  x  0.0001489)  (number  of  estimated  students). 


Table  3.2 

Standard  Regression  Model  with  FMS  Composite  Predicting  Student  Load 


Constant 

Variable 

B 

SE 

Adjusted  R2 

F 

71.22 

FMS 

.0001489 

.00000810 

.91 

337.98* 

*p  <  .001. 


Establish  a  Requirements  Review  Process 

As  previously  discussed,  AF/A1DG  was  developing  a  process  to  determine  requirements  at  the 
time  of  this  study.  Newly  assigned  personnel  were  aggressively  seeking  to  correct  deficiencies 
in  the  requirement  process  of  past  years  and  to  advocate  for  DLIELC  resources.  But  a  process 
needs  to  be  defined  in  policy,  and  the  executive  agent  (EA) — a  role  assumed  by  AF/A1DG — 
needs  to  have  the  authority  to  force  the  process.  The  EA  is  the  key  to  making  a  requirements 
process  work.  The  EA  must  lead  the  process.  Figure  3.6  provides  a  simple  framework  for  a 
requirements  process. 
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Figure  3.6 

A  Basic  Framework  for  Determining  Requirements 
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The  EA  provides  the  initial  starting  point.  In  the  discussion  above,  we  provide  a  for¬ 
mula  to  estimate  the  demand  by  COCOM.  This  is  a  justifiable  starting  point.  The  EA  then 
establishes  deadlines  for  the  COCOMs  and  military  departments  to  refine  the  estimates  with 
justifiable  arguments.  The  EA  consolidates  all  the  requirements  and  eliminates  duplication  to 
create  one  requirement  document.  DLIELC  is  an  executing  agency  and  not  the  user;  there¬ 
fore,  it  is  not  involved  in  the  process  up  to  this  point.  Having  defined  a  requirement,  the  EA 
tasks  DLIELC  to  match  capacity  to  the  demand.  If  there  are  disconnects,  the  EA  can  work 
with  DLIELC  to  execute  options  (e.g.,  temporary-contracted  ELT)  and  plan  for  additional 
capacity  if  warranted.  The  constrained  requirement  goes  back  to  the  COCOMs  and  military 
departments  for  a  final  look  to  determine  if  readjusting  start  dates  or  using  in-house  training 
capacity  can  satisfy  some  of  the  need.  The  EA  takes  the  final  product  and  develops  a  schedule 
in  conjunction  with  DLIELC.  While  countries  seldom  seem  to  follow  a  strict  schedule,  they 
do  roughly  follow  one  to  meet  FOT  dates. 


Managing  Demand 

The  ability  to  forecast  how  many  students  will  come  to  DLIELC  at  a  given  time  is  critical,  but 
knowing  how  long  they  will  remain  in  school  and  how  to  program  their  training  is  equally 
important. 

Programming  Students 

Students  are  tested  in-country14  for  their  ECL  and  retested  during  their  first  week  of  train¬ 
ing.  To  ensure  that  students  will  not  lose  their  English  skills  while  they  are  waiting  to  come 
to  DLIELC,  there  is  a  maximum  delay  of  105  days  between  the  time  a  student  is  tested  in¬ 
country  and  his/her  arrival  in  San  Antonio.  This  second  score,  as  well  as  the  ECL  score  required 
by  their  FOT,  serves  as  the  basis  for  programming  students  for  an  appropriate  number  of  weeks 
and  for  placing  them  in  the  relevant  class  level.  Although  the  programming  wheel15  is  the  pri¬ 
mary  tool  to  determine  training  time,  the  amount  of  funding  available  for  individual  students 
may  also  influence  how  much  training  time  is  ultimately  programmed. 

Forecasting  the  precise  training  time  that  students  will  need  to  achieve  their  target  ECL 
has  generally  proved  challenging.  Programmed  training  time  can  be  highly  variable  for  GET 


14  In-country  refers  to  the  home  nation  of  the  student. 

15  The  wheel  is  a  table  of  ECL  scores  correlated  to  the  required  length  of  training,  which  also  determines  the  starting  book 
number. 
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students  (from  one  to  96  weeks  in  FY01-09).16  Programmed  training  time  is  routinely  reas¬ 
sessed  during  the  training,  with  students  being  granted  extensions  and  curtailments  when 
needed.  Figure  3.7  shows  that  the  average  actual  training  time  has  increased  over  the  past  ten 
years  but  remained  within  a  two-week  range  (from  13  weeks  per  student  in  FY01  to  15  weeks 
in  FY09.17)  Similarly,  the  average  length  of  extensions  and  curtailments  has  remained  within  a 
two-week  range  in  FY06-09. 

Averages,  however,  conceal  important  variations  across  individuals.  Extensions  ranged 
from  one  to  76  weeks  (standard  deviation  of  9.7).  The  average  curtailment  was  seven  weeks 
but  ranged  from  one  to  80  weeks  (standard  deviation  of  8.5).  The  training  time  is  made  more 
unpredictable  by  the  fact  that  the  percentage  of  IMSs  requiring  extensions  and  curtailments 
has  increased  over  the  years.  In  FY06,  6.9  percent  of  all  IMSs  received  an  extension,  increasing 
to  11.7  percent  in  FY09.  In  FY06,  13.2  percent  of  IMSs  received  a  curtailment,  increasing  to 
16.9  percent  in  FY09. 

To  complicate  planning  further,  not  all  students  need  to  wait  until  they  ECL-qualify  (i.e., 
reach  the  necessary  established  standard  to  continue  on)  to  go  to  FOT:  In  2010,  96  students 
received  a  waiver  that  lifted  this  requirement.  However,  the  percentage  of  IMSs  who  receive 
waivers  remains  limited,  never  exceeding  3  percent  of  all  IMSs  in  2005-2009. 

Finally,  although  DLIELC  students  can  graduate  at  any  given  week,  they  do  not  neces¬ 
sarily  leave  the  school  once  they  reach  their  target  ECL,  sometimes  remaining  at  DLIELC  for 

Figure  3.7 

Ideal,  Programmed,  and  Actual  Training  Time  per  IMS,  FY01-FY09 


NOTE:  Ideal  is  the  amount  of  training  time  that  a  student's  ECL  scores  say  will  be  required  (requires  use  of  the 
wheel).  The  programmed  training  time  is  the  amount  programmed  by  the  organization.  The  actual  training  time 
is  what  students  used.  The  chart  highlights  the  need  to  update  the  wheel  to  more  accurately  predict  training  time. 

RAND  TR1152-3.7 


16  SET  and  U.S.  students  are  systematically  programmed  for  nine  and  24  weeks,  respectively. 

17  These  figures  include  U.S.  students. 
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several  weeks  until  they  can  move  on  to  their  FOT.  In  this  interim  period,  DLIELC  is  respon¬ 
sible  for  the  well-being  of  these  individuals,  although  the  school  has  yet  to  develop  innovative 
programs  to  use  their  time  constructively.  Whether  these  incremental  costs  to  the  organiza¬ 
tion  are  accurately  captured  and  reimbursed  is  unlikely,  given  the  underdeveloped  state  of  the 
organization’s  costing  models.  Students  are  sent  back  to  classes  they  have  already  attended — 
sometimes  several  levels  down  from  the  last  one  they  attained — to  continue  practicing  their 
English  language  skills.  Although  this  method  does  keep  students  busy,  it  consumes  school 
resources,  especially  teaching  time.  It  is  also  likely  that  rehashing  a  previously  studied  book 
results  in  poor  motivation  and  attention  in  class,  potentially  making  the  work  of  the  teaching 
staff  more  difficult.  According  to  DLIELC,  these  students  can  be  particularly  problematic: 
“They  lose  their  English,  start  having  more  disciplinary  issues,  disturb  other  students.”18  In 
2009,  almost  20  percent  of  IMSs  started  their  FOT  four  to  eight  weeks  after  they  graduated 
(Figure  3.8). 

ELT  Length  for  New  Countries 

The  length  of  English  training  is  another  unmanageable  factor.  Initial  and  target  ECL  scores 
provide,  through  programming  benchmarks  (i.e.,  the  “wheel”),  the  predicted  training  time  a 
student  needs  to  complete  ELT.  In  cases  where  students  of  a  country  have  attended  DLIEC 
before,  the  average  length  of  training  can  be  estimated  by  the  country  average.  When  a  new 
country  requests  or  requires  ELT,  it  is  possible  to  use  world  modernity  data19  to  estimate  an 
average  ECL  score. 

Figure  3.8 

Time  Between  Graduation  and  FOT 


30 


After  FOT  Less  than  1  to  2  2  to  4  4  to  8  8  to  1 2  More  than 


starts  1  week  weeks  weeks  weeks  weeks  12  weeks 
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18  Focused  discussion  with  AF/A1DG,  October  22,  2010. 

19  Modernity  data  attempt  to  measure  the  advancement  of  a  country  by  multiple  types  of  data.  Key  areas  include  education, 
the  role  of  females,  infrastructure,  commerce,  birth  rates,  poverty,  literacy,  Internet  usage,  government,  and  many  other 
factors. 
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The  eight  variables  in  Table  3.3  were  able  to  explain  43  percent  of  the  variation  in  ECL 
scores  by  country.20  While  the  estimate  is  not  perfect,  it  is  a  starting  point  for  estimating  a  new 
FMS  program’s  required  ELT  costs  (see  Table  3.4). 


Table  3.3 

Standard  Regression  of  Selected  Modernity  Data  Predicting  ECL  Score 


Factor 

Estimate 

Standard 

Error 

T  Ratio 

Prob  >|t| 

Intercept 

65.11 

4.19 

15.54 

0.0000 

Outpatient  visits  per  capita 

-0.63 

0.37 

-1.71 

0.0908 

Technicians  in  R&D  (per  million  people) 

0.01 

0.00 

2.24 

0.0273 

International  tourism,  expenditures  for  passenger 
transport  items  (current  US$) 

-0.00 

0.00 

-2.08 

0.0407 

Trained  teachers  in  primary  education,  male 
(%  of  male  teachers) 

-0.09 

0.03 

-3.47 

0.0008 

Expenditure  per  student,  primary 
(%  of  GDP  per  capita) 

0.47 

0.13 

3.49 

0.0008 

Roads,  paved  (%  of  total  roads) 

-0.11 

0.04 

-2.91 

0.0045 

Liner  shipping  connectivity  index 
(maximum  value  in  2004  =  100) 

0.27 

0.08 

3.55 

0.0006 

Ratio  of  female  to  male  secondary  enrollment  (%) 

0.13 

0.05 

2.89 

0.0048 

Table  3.4 

Analysis  of  Variance  for  Predicting  ECL  Score 


Sum  of  Squares 
Due  to  Error 

DFE 

MSE 

R2 

Adjusted  R2 

Prob  >  F 

9956.0 

103 

96.7 

.473 

.427 

<.0001 

NOTES:  DFE  =  degrees  of  freedom;  MSE  =  mean  square  error. 


Medical  Clearances 

Medical  clearances  represent  another  planning  and  scheduling  issue.  Numerous  types  of  FOT 
require  foreign  students  to  obtain  a  certificate  stating  that  they  fulfill  all  medical  requirements 
for  their  FOT,  and  this  process  can  prove  particularly  time-consuming,  especially  when  the 
students’  country  of  origin  does  not  provide  generalized  or  quality  medical/dentistry  care.  In 
some  cases,  students  are  unable  to  obtain  the  required  medical  clearance,  and  their  EFT  time 
is  cut  short  by  elimination.21  In  other  cases,  students  obtain  the  medical  clearance  but  only 
after  spending  more  time  at  DFIEFC  than  required  for  their  English  language  training.  As  of 
December  31,  2010,  fewer  than  70  percent  of  DFIEFC  students  who  needed  a  medical  clear¬ 
ance  managed  to  obtain  one.22 


20  More  than  206  variables  were  used  to  screen  for  variables  with  some  correlation  to  predicting  an  ECL  score.  The  eight 
variables  were  found  to  be  statistically  significant  predictors. 

21  In  some  instances,  the  FOT  slot  is  transferred  to  another  student  who  holds  a  medical  clearance. 

22  DLIELC,  Balanced  Scorecard  FY10-3,  undated. 
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Managing  Supply 

Before  we  can  conclude  that  demand  exceeds  supply,  we  must  carefully  examine  supply.  Supply 
comprises  a  number  of  factors.  Some  of  the  more  significant  are  classrooms,  instructors,  beds, 
parking  spaces,  dining  capacity,  and  course  material.  There  are  other  factors,  but  these  are  the 
primary  ones  we  addressed,  and  among  this  list  we  focused  on  classrooms.  We  let  classrooms 
drive  the  number  of  instructors  required.  Physical  space  (classrooms)  has  the  longest  lead  time 
and  requires  MILCON  funding.23  Instructors  are  driven  by  the  demand  for  classrooms.24 

We  did  not  model  certain  items.  Beds  (housing)  are  purchasable  off  base  and  therefore 
do  not  represent  a  hard  constraint.  Parking  spaces  were  addressed  by  the  schoolhouse  (there 
is  sufficient  space  to  expand).  Dining  capacity  had  room  for  another  shift  (thus,  we  did  not 
model  it).  Course  materials  were  driven  by  demand.  (We  assumed  these  were  a  cost  item  and 
not  a  restriction.) 

We  created  a  simulation  model  of  the  schoolhouse  processes.  The  model  is  a  simple  pro¬ 
duction  model  with  individuals  progressing  in  the  course  and  week  of  instruction  where  the 
input  is  the  historical  entry  of  students.  There  is  a  stochastic  entry  for  failures.  At  the  end  of 
each  week  (the  unit  of  measure  in  the  model),  the  students  are  placed  in  new  classes  (commen¬ 
surate  to  their  course’s  MASL  and  week  of  study),  with  the  goal  of  maintaining  a  6:1  ratio  of 
students  to  instructors.  If  the  requirement  exceeds  capacity,  the  ratio  is  increased  up  to  a  speci¬ 
fied  limit.  Every  effort  is  made  to  eliminate  classes  of  one  or  two  students.  Still,  we  found  that 
human  intelligence  can  do  a  better  job  of  distributing  the  students  than  the  model  can,  which 
is  why  our  model  replicated  overall  demand  but  used  more  classrooms  than  were  used  histori¬ 
cally.  For  this  reason,  the  model  represents  an  absolute  requirement  or  worst  case  and  not  the 
actual  distribution  of  students  into  classrooms.  Suboptimal  actions,  such  as  holding  students 
back  (who  would  have  proceeded)  or  pushing  them  forward  can  reduce  the  demand  on  class¬ 
rooms,  but  they  depend  on  teacher  evaluations  of  the  students. 

Model  results  are  shown  in  Figure  3.9;  a  peak  demand  (modeled  by  historical  entries) 
occurs  every  year  between  May  and  July  and  stresses  the  system.  The  peak  demand  is  driven 
primarily  by  professional  military  education  (PME),  which  starts  in  the  fall. 

Seasonal  peaks  represent  a  major  planning  challenge  for  DLIETC.  The  fact  that  these 
surges  in  student  load  are  not  permanent  makes  it  difficult  to  request  additional  funds  based 
on  them,  because  extra  teaching  staff  and  new  infrastructure  would  only  be  used  during  this 
peak  time — and  remaining  largely  idle,  and  empty,  during  nonpeak  times. 

Classrooms 

Simulation  runs  suggest  that  delaying  admittance  of  students  without  FOT  can  reduce  the 
peaks  that  occur  in  May  through  July.  This  effect  was  less  noticeable  in  2010,  but  was  notice¬ 
able  in  the  years  prior  to  2010. 

Prioritization  is  another  possible  approach  to  reducing  the  peak  demand,  although  it  does 
have  negative  aspects.  If  students  could  be  prioritized  in  such  a  way  as  to  leave  lower-priority 
students  out  of  the  peak  period,  demand  could  be  significantly  reduced.  Unfortunately,  some 
FOT  courses,  such  as  PME,  have  very  specific  start  times.  Figure  3.10  shows  the  effect  of  delay- 


23  The  analysis  is  not  limited  to  in-house  options.  Later  in  the  report,  we  look  at  contracted  options. 

24  The  model  estimates  the  number  of  classrooms.  We  assumed  one  instructor  per  classroom.  Therefore,  the  demand  for 
instructors  is  driven  by  the  demand  for  classrooms. 
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Figure  3.9 

Simulated  Classroom  Demand,  2005-2010 


2005  2006  2007  2008  2009  2010 
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Figure  3.10 

Effect  of  Delaying  Students  Without  FOT 


2005  2006  2007  2008  2009  2010 

NOTES:  Input  data  for  this  figure  were  supplied  by  DLIELC.  The  RAND  simulation  model  delayed  entries  based  on 
the  existence  of  FOT. 
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ing  30  percent  of  all  students  and  half  of  the  students  without  FOT.  Some  peaks  are  reduced; 
in  other  cases,  the  overall  demand  is  spread  out  more. 

The  simulation  model  shows  a  requirement  over  the  past  five  years  of  200  classrooms.  The 
limit  is  reached  only  three  times,  and  only  during  the  peak  periods.  Also,  the  model  is  based 
on  an  absolute  requirement  and  does  not  model  the  hand-optimization  that  occurs  at  DLIELC 
when  small  classrooms  (one  to  three  students)  are  merged  with  other  classrooms.25  The  simu¬ 
lation  shows  that  capacity  exceeds  the  current  150  classrooms  most  of  the  time.  DLIELC’s 
internal  reorganization  of  space  to  reach  174  is  also  breached  most  of  the  time.  For  these  rea¬ 
sons,  200  classrooms  would  seem  to  be  a  reasonable  goal  and  is  an  increase  of  only  26  over  the 
proposed  174-classroom  modification.26 

Another  solution  would  be  to  avoid  scheduling,  during  peak  months,  students  who  plan 
to  attend  frequently  occurring  FOT  sessions.  These  students  represent  the  next  level  of  flex¬ 
ibility;  they  do  have  FOT,  but  they  could  possibly  have  their  start  times  rescheduled  so  they 
would  not  coincide  with  FOT  sessions  that  have  one  start  date  per  year  or  that  begin  with  the 
academic  year. 

Currently,  an  increase  in  Saudi  students  is  being  discussed  that  will  greatly  exceed  the 
growth  of  the  past  five  years,  possibly  doubling  the  size  of  the  school.  There  are  two  problems 
with  building  to  such  capacity.  First,  the  growth  in  DLIELC  has  been  gradual  over  the  past 
five  years,  with  one  year  actually  witnessing  a  decrease.  Second,  there  is  no  guarantee  that, 
after  this  expected  surge  in  growth  over  the  next  two  to  three  years,  the  demand  will  continue 
and  not  fall  back  to  pre-surge  conditions.  The  upcoming  demand  appears  to  be  an  exception. 
In  that  case,  temporary  surge  options  make  more  sense,  unless  a  specific  growth  in  the  FMS 
budget  is  being  forecast,  which  it  is  not.  Therefore,  we  recommend  the  use  of  contracted  ELT 
services  over  the  next  three  years  to  cover  the  expected  surge,  as  opposed  to  building  a  facility 
(beyond  the  200-classroom  requirement)  that  may  not  be  fully  utilized  after  the  surge  time 
frame  has  passed. 

The  challenge  for  students  who  cannot  start  FOT  right  after  they  reach  their  target  ECL 
is  to  maintain,  during  this  transition  time,  the  English  language  skills  they  have  acquired.  The 
current  practice  of  sending  them  to  classes  they  have  already  attended  consumes  resources  and 
is  educationally  ineffectual. 

One  potential  solution  would  be  to  set  up  a  “finishing  school”  for  these  students.  This 
would  isolate  potentially  disruptive  and  less-motivated  students  from  those  who  are  still  work¬ 
ing  on  trying  to  achieve  their  ECL.  It  would  also  provide  an  opportunity  to  tailor  ELT  to 
the  specific  needs  of  students  who  are  about  to  start  FOT,  emphasizing  those  skills  that  FOT 
instructors  have  repeatedly  identified  as  lacking  among  DLIELC  students.  Feedback  from  field 
evaluation  questionnaires  (FEQs)27  indicates  that  these  skills  include  knowledge  of  military 
jargon  and  acronyms,  understanding  of  fast-paced  speech,  and  familiarity  with  various  regional 
accents.  Setting  up  a  finishing  school  would  require  fewer  teaching  staff  than  if  students  were 
put  back  in  regular  classrooms.  Training  could  also  be  more  self-paced,  self-directed,  and  make 


25  Again,  these  are  suboptimal  allocations  as  students  are  moved  forward  or  back  to  create  room.  Also,  the  merging  of  class¬ 
rooms  represents  the  merging  of  students.  The  rooms  are  fixed  in  size  (maximum  of  ten  students)  and  location. 

26  As  of  February  23,  2011,  DLIELC  was  planning  an  in-house  increase  of  24  classrooms  (DLIELC  English  Language 
Program  Working  Group  Briefing,  December  9,  2010). 

27  AETC  sends  FEQs  typically  to  AF  units  to  evaluate  the  quality  of  training  in  new  airmen.  In  this  case,  FEQs  are  sent  to 
the  follow-on  schoolhouses  for  evaluation  of  the  English  capability  of  the  students. 
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more  extensive  use  of  computer  resources  instead  of  instructors.  And  a  finishing  class  would 
also  free  up  space  in  instructional  classes,  which  would  further  support  the  desired  6:1  student- 
to-instructor  ratio.  These  classes  may  even  be  designed  with  up  to  20-25  students  per  instruc¬ 
tor,  if  group-based  activities  are  employed. 

The  option  of  a  finishing  school  class  would  be  a  good  place  to  use  U.S.  military  reservists 
or  retirees  to  expose  IMSs  to  military  jargon  and  slang. 

Another  option  would  be  to  turn  students  who  are  ECL-qualified  into  teaching  assistants 
(TAs).28  This  would  increase  teaching  staff  while  keeping  these  students  busy  practicing  their 
English.  The  “promotion”  represented  by  this  TA  position  would  also  contribute  to  keeping 
these  students  motivated  by  putting  them  in  the  position  where  they  are  transmitting  their 
knowledge  to  other  students.  For  that  same  reason,  it  could  considerably  reduce  discipline 
issues.  This  solution  could  work  particularly  well  for  students  who  have  a  relatively  long  wait¬ 
ing  time  before  starting  their  FOT  (two  weeks  or  more). 

Instructors 

While  not  affecting  the  number  of  classrooms  available,  restricting  instructor  leave  during  the 
peak  window,  as  a  university  or  technical  school  would  do,  reduces  the  student-to-teacher  ratio 
by  8  percent.  Figure  3.11  shows  the  effect  of  this  option  and  others  considered  below. 

Institutionalizing  a  dual-role  curriculum  development  section  in  which  individuals  are 
required  to  teach  during  peak  periods  can  reduce  student-to-teacher  ratios  by  a  full  student 
(e.g.,  7:1  to  6:1). 

Fliring  temporary  instructors  during  the  peak  period  reduces  the  ratio  significantly  but  is 
costly  because  the  shortest  currently  available  temporary  program  is  six  months.29  A  three-  to 
four-month  temporary  hiring  contract  would  work  the  best  to  reduce  peak  demand  and  would 
not  result  in  overcapacity. 

Nonresident  training  represents  another  way  to  add  flexibility  to  planning.  To  retain 
more  instructors  at  DFIEFC  when  the  school  needs  them  most — i.e.,  during  peak  times — one 
solution  would  be  to  schedule  only  MTTs  deemed  critical  to  COCOM  theater  security  coop¬ 
eration  program  (TSCP)  priorities  during  peak  months  and  delay  other  MTTs  to  nonpeak 
months,  hack  of  access  to  nonresident  training  data  has  prevented  us  from  examining  precisely 
how  many  instructors  such  a  measure  could  free  up,  but  this  possibility  should  be  explored  in 
case  the  other  options  fail  to  reduce  seasonal  peaks. 

Relocating  ECHO  Company 

Another  way  to  create  some  breathing  space  at  DFIEFC  would  be  to  focus  on  IMSs  at  Fack- 
land  Air  Force  Base  in  San  Antonio  and  relocate  ECHO  Company.30  IMSs  and  ECHO  Com¬ 
pany  currently  share  some  infrastructure,  including  classrooms,  a  dining  facility  (the  Alamo 
Cafe),  and  a  library,  but  they  attend  different  classes  and  language  labs.  They  are  also  billeted 


28  We  note  that  the  terms  “finishing  school”  and  “TA”  might  need  to  be  defined  and  marketed  differently  to  ensure  that 
cultural  and  linguistic  sensitivities  are  fully  considered.  The  best  solution  to  these  issues  will  likely  be  determined  by 
DLIELC. 

29  These  results  are  based  on  a  General  Services  Administration  contract  that  DLIELC  was  using  at  the  time.  A  DLIELC- 
initiated  IDIQ  contract  may  be  able  to  reduce  the  temporary  instructor  time  to  four  months. 

30  ECHO  Company  is  a  U.S.  Army-only  company  of  recruits,  deficient  in  English,  who  learn  English  prior  to  attendance 
at  Army  basic  training. 
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Figure  3.11 

Policy  Effects  on  Student-to-Teacher  Ratio 
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in  separate  buildings.  While  IMSs  study  in  a  mostly  civilian  environment,  U.S.  students  expe¬ 
rience  a  more  military  environment,  where,  as  indicated  by  a  DLIELC  source,  “every  waking 
moment  is  controlled  by  U.S.  Army  drill  sergeants.”31  U.S.  students  receive  a  different  training 
than  IMSs:  “[D]rill  sergeants  teach  them  military  skills  such  as  basic  rifle  marksmanship,  land 
navigation,  first  aid,  etc.,  since  they  do  not  get  ‘free  time’  like  the  IMS.”32  International  and 
U.S.  students  have  different  initial  ECL  scores,  training  times,  and  success  rates  (defined  as  the 
percentage  of  students  who  reach  their  target  ECL).33  Overall,  DLIELC  caters  to  two  cohorts 
that  have  very  different  profiles  and  needs.  Relocating  ECHO  Company  would  free  up  some 
teaching  staff,  classrooms,  and  dorm  space  at  Lackland  Air  Force  Base,  which  would  greatly 
help  DLIELC  in  absorbing  increasing  numbers  of  IMSs. 

Additionally,  assuming  that  facilities  exist  at  such  locations  as  Fort  Jackson,  relocating 
ECHO  Company  would  save  some  costs  associated  with  shipping  graduates  to  their  basic  mili¬ 
tary  training  (BMT)  base.  If  the  new  location  is  a  BMT  base  like  Fort  Jackson,  those  students 
going  to  Fort  Jackson  for  BMT  would  have  no  costs,  but  other  students  would  have  travel  costs 
similar  to  those  for  shipping  from  San  Antonio. 

Improve  Programming  by  Recalibrating  the  Wheel 

Programming  is  largely  based  on  the  wheel,  which  predicts  training  timelines  based  on  initial 
ECL  scores.  However,  initial  ECL  scores  predict  only  37  percent  of  the  variance  in  time  to 
ECL-qualify.  The  discrepancy  between  average  programmed  time  and  actual  training  time 


11  Email  correspondence  with  DLIELC,  January  21,  2010. 

32  Email  correspondence  with  DLIELC,  January  21,  2010. 

33  This  gap,  however,  has  been  greatly  reduced  since  FY08. 
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remains  small  (within  a  one-week  range  in  FY01-09),  but  this  conceals  important  variations 
across  individuals. 

Overall,  in  FY09,  12  percent  of  IMSs  were  underprogrammed  (i.e.,  had  their  training 
time  extended)  while  17  percent  were  overprogrammed  (i.e.,  their  training  time  was  cur¬ 
tailed).  Eight  percent  of  U.S.  students  received  an  extension;  96  percent  had  their  training 
time  curtailed. 

Students  can  be  underprogrammed  for  a  number  of  reasons.  In  some  instances,  students 
turn  out  to  be  illiterate  in  their  own  language,  making  the  learning  of  a  second  language  par¬ 
ticularly  challenging.  According  to  a  DLIELC  source: 

We  currently  do  not  have  screening  in  place  at  DLI  [Defense  Language  Institute]  to  deter¬ 
mine  literacy  in  a  student’s  native  language,  and  students  may  or  may  not  admit  this  defi¬ 
ciency.  Once  students  are  here,  they  want  to  save  face,  and  most  work  hard,  but  research 
indicates  that  second  language  acquisition  for  an  adult  who  is  not  literate  in  his/her  native 
language  is  very  difficult,  if  not  impossible.34 

Another  reason  for  underprogramming  is  that  “[l]ack  of  sufficient  funding  may  preclude 
the  student  from  being  scheduled  for  the  correct  number  of  weeks  according  to  DLIELC  pro¬ 
gramming  guidelines.”35  In  the  latter  case,  lack  of  funding  means  that  students  will  leave  the 
school  without  having  reached  their  target  ECL. 

In  other  instances,  some  students  are  adequately  programmed  based  on  their  capabilities 
but  are  still  not  able  to  finish  in  time.  According  to  DLIELC  sources,  “[e]ven  when  a  student 
has  been  accurately  programmed  in  accordance  with  DLIELC  guidelines,  external  factors  such 
as  visa  delays  may  result  in  a  discrepancy  between  programmed  and  actual  training  time.”  In 
some  instances,  students  have  difficulties  studying  and  completing  their  program  because  of 
“excessive  absences  due  to  personal  or  dependent  care.”  Other  factors  explaining  student  failure 
include  “lack  of  linguistic  aptitude,  lack  of  motivation,  age,  .  .  .  and  culture/values  adjustment 
time.”  Finally,  in  some  instances,  some  students  seem  to  try  to  “game  the  system”  by  purposely 
underperforming  at  tests  in  order  to  stay  longer  at  DLIELC.36 

IMSs  and  U.S.  students  who  receive  curtailments  generally  do  so  because  they  have 
reached  their  target  ECL  early.  In  the  case  of  U.S.  students,  however,  other  factors  come  into 
play.  In  2005-2010,  87  percent  of  U.S.  students  who  did  not  ECL-qualify  were  curtailed  by 
one  week  or  more,  largely  because  of  “academic  failure,  disciplinary  action,  failure  to  adjust  to 
military  life  or  other  UCMJ  [Uniform  Code  of  Military  Justice]  punishable  offenses.”37  These 
curtailments  often  occur  early  in  the  program:  In  2005-2010,  11  percent  of  all  U.S.  students 
were  curtailed  by  16  weeks  or  more. 

An  initial  ECL  score  should  not  be  the  only  relevant  factor  to  program  training  time. 
Native  language  literacy  rate,  student  motivation,  and  other  factors  identified  in  empirical  lit¬ 
erature  on  learning  a  second  language  also  play  an  important  role.  One  option  for  DLIELC 


34  Email  correspondence  with  DLIELC,  September  16,  2010. 

35  Email  correspondence  with  DLIELC,  September  16,  2010. 

36  Focused  discussion  with  AF/A1DG,  October  22,  2010. 

37  Email  correspondence  with  DLIELC,  December  15,  2010.  IMSs,  however,  are  very  rarely  dismissed  for  dis¬ 
ciplinary  issues,  and  testing  their  ability  to  adapt  to  military  life  is  not  the  purpose  of  the  school  for  them. 
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would  be  to  recalibrate  its  programming  wheel  by  combining  an  initial  ECL  score  with  these 
factors  to  accommodate  different  rates  of  learning. 

Because  native  language  literacy  rate  seems  to  be  a  particularly  important  predictor  of 
how  well  students  will  perform  in  ELT,38  another  option  could  be  to  screen  applicants  in¬ 
country  by  giving  them  a  Defense  Language  Aptitude  Battery  (DLAB)  test.  DLAB  tests  mea¬ 
sure  how  capable  a  student  will  be  to  learn  another  language  (e.g.,  English) — rather  than 
simply  assessing  how  well  he/she  already  speaks  it.  This  has  already  been  implemented  for  a 
very  small  subset  of  students:  DLIELC  informed  us  that  “[cjurrently  work  is  being  done  at  the 
Center  for  Advanced  Study  of  Language  (CASL)  at  the  University  of  Maryland  to  develop 
a  DLAB  test  for  Afghan  students.”39  This  process  should  be  generalized  to  avoid  sending  to 
DLIELC  students  who  realistically  have  very  little  chance  of  succeeding. 

Recalibrating  the  wheel  would  require  identifying  (1)  the  factors  beyond  ECL  initial 
score  that  impact  a  student’s  training  time,  (2)  whether  it  is  practical  to  measure  these  factors, 
and  (3)  whether  their  effects  are  significant.  These  factors  could  then  be  integrated  in  a  tool 
that  would  ideally  be  as  user  friendly  as  the  current  wheel.  If  this  effort  proves  too  complex, 
a  simple  way  to  improve  the  wheel  would  be  to  add,  as  a  sole  additional  consideration,  stu¬ 
dents’  literacy  in  their  own  language.  This  would  require  one  additional  in-country  test  for  all 
students.  Although  this  may  consume  resources,  a  wheel  incorporating  both  factors — initial 
ECL  score  and  native  language  literacy — is  likely  to  provide  more-accurate  training  times  for 
student  programming. 

Near-Term  Strategy  for  Supply 

A  near-term  strategy  will  require  addressing  classroom  space,  the  number  of  instructors,  and 
how  the  “peak”  period  is  managed.  It  is  inefficient  to  hire  government  employees  for  the  peak 
workload  requirement.  If  we  assume  that  an  IDIQ  contracting  vehicle  can  provide  instructors 
for  short  periods  of  time  (4-6  months)  and  that  the  cost  of  an  IDIQ  instructor  is  less  than  a 
full-time  government  employee  (because  the  instructor  works  less  than  half  the  year),  and  if  we 
also  assume  that  a  contracted  option  (full  ELT  training  to  include  curriculum,  materials,  facili¬ 
ties,  and  lodging  at  another  location)  is  more  expensive  than  the  current  government  option 
(utilizing  the  opportunity  costs  of  some  existing  infrastructure),  then  we  believe  the  most  cost- 
effective  option  is  as  follows: 

•  Build  capacity  to  200  rooms  and  hire  government  instructors  to  meet  the  current  low 
demand  of  160  rooms  (assuming  that  instructor  unavailability  and  other  requirements 
yield  a  need  for  175  instructors). 

•  Manage  the  additional  surge  of  40  instructors  via  an  IDIQ  contract. 

•  Establish  an  off-site  contract  option  for  unforeseen  demand  that  could  handle  as  many  as 
50  additional  classrooms  on  an  as-needed  basis. 

•  Utilize  this  contract  at  a  low-demand  rate  (5-10  classes)  to  maintain  its  availability  and 
rapid  response. 

•  A  final  option  for  peak  demand  might  be  to  utilize  qualified  reservists  as  instructors.  It 
would  have  the  added  benefit  of  exposing  IMSs  to  U.S.  military  personnel. 


38  This  result  is  based  on  discussions  with  DLIELC  sources.  DLIELC  had  no  data  available  on  students’  literacy  in  their 
own  language,  making  it  impossible  for  the  RAND  team  to  examine  how  lack  of  literacy  affects  success  at  DLIELC. 

39  Email  correspondence  with  DLIELC,  September  16,  2010. 
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This  strategy  is  illustrated  in  Figure  3.12.  The  solid  line  is  the  historical  modeled  demand 
for  classrooms.  The  solid  blocks  are  the  suggested  in-house  instructor  capacity.  This  keeps  the 
main  cadre  of  instructors  sufficiently  utilized  at  the  least  cost.  The  dashed  blocks  on  top  repre¬ 
sent  the  IDIQ  temporary  contractors  who  are  hired  on  a  six-month  window  to  meet  demand 
requirements.  For  requirements  outside  of  the  blocks  or  above  the  200-classroom  requirement, 
outside  contractors  can  be  utilized  until  the  demand  is  confirmed  as  a  long-term  demand,  and 
additional  facilities  can  be  purchased. 


Case  Study  Observations:  A  Strategic  Prioritization-Driven  Approach 

Several  of  the  organizational  case  studies  that  RAND  reviewed  for  this  report  have  transi¬ 
tioned  from  a  supply-driven  approach  to  a  strategic  prioritization-driven  approach  to  create 
and  maintain  security  cooperation  programs.  This  shift  in  planning  and  resourcing  in  these 
organizations  is  partly  tied  to  new  leadership  and  guidance,  rigorous  outreach  efforts  to  the 
COCOMs,  the  institution  of  policy-oriented  steering  groups  to  advise  and  guide  the  organi¬ 
zation’s  activities,  and  the  recognition  of  the  need  to  work  more  closely  with  intergovernmen¬ 
tal  and  nongovernmental  organizations.  For  example,  the  Center  of  Excellence  for  Disaster 
Management  and  Humanitarian  Assistance  (DMHA)  has  representatives  at  several  geographic 
COCOMs  and  understands  well  that  partnering  with  established  intergovernmental  and  non¬ 
governmental  organizations  in  the  respective  partner  countries  can  have  mutually  beneficial 
outcomes.  The  Defense  Institute  for  International  Legal  Studies  (DHLS)  has  shifted  from  a 
prior  business  model  based  on  “unconstrained  growth”  to  one  that  emphasizes  country  stra- 
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tegic  priorities  as  provided  by  the  Office  of  the  Secretary  of  Defense  for  Policy,  (OSD(P))  in 
the  Guidance  for  Employment  of  the  Force  (GEF).  DHLS  has  already  established  a  Policy 
Oversight  Council,  which  began  work  in  fall  2010.  Both  the  Center  for  Excellence  in  Disaster 
Management  and  Humanitarian  Assistance  (COE-DMHA)  and  DHLS  have  close  linkages 
to  DSCA  and  to  OSD(P),  which  facilitates  connections  to  policy  decisions  and  current  guid¬ 
ance  on  country  and  activity  prioritization.  What  both  these  organizations  have  found  to  be  of 
critical  importance  in  developing  these  new  models  is  a  flexible  approach  to  the  organization’s 
management,  access  to  a  large  labor  pool  of  talent,  and  the  ability  to  put  labor  overage  on  a 
viable  contract  vehicle. 

Several  organizations  that  RAND  examined,  including  the  IAAFA,  the  Defense  Insti¬ 
tute  for  Medical  Operations  (DIMO),  and  the  Partner  Language  Training  Center,  Europe 
(PLTCE),  have  adapted  their  programs  and  activities  to  better  align  themselves  with  strategic 
guidance  and  to  better  meet  the  needs  of  the  partner  nations.  Specifically,  they  have  varied 
course  structure  and  lengths  and  increased  use  of  mobile  training  teams  in  recent  years. 


Recommendations 

As  noted  above,  we  find  that  DLIELC  lacks  an  overall  robust  requirements  determination  pro¬ 
cess  and  that  the  organization’s  business  model  is  unable  to  meet  variable  demand.  The  more 
specific  findings  within  this  larger  finding  and  potential  solutions  described  above  can  be  sum¬ 
marized  in  the  following  set  of  recommendations:40 

•  Institute  processes  to  enable  more  accurate  depiction  of  demand. 

-  Identify  factors  to  recalibrate  the  training  wheel  and  more  accurately  program  students. 

-  Use  regression  models  to  predict  student  load  based  on  estimates  of  FMS. 

-  Use  regression  models  to  predict  average  ELT  time  for  new  countries  based  on  world 
modernity  data. 

•  Collaborate  with  the  Defense  Language  Testing  Advisory  Board  to  fully  review  the  valid¬ 
ity  and  reliability  of  the  ECL.  This  review  will  help  ensure  that  students  are  accurately 

programmed,  placed  into  the  right  course  level,  and  sufficiently  skilled  to  attend  FOT. 

•  Reduce  seasonal  peaks. 

-  Schedule  students  with  no  FOT  during  nonpeak  months. 

-  Avoid  scheduling  during  peak  months  students  who  plan  to  attend  a  frequently  occur¬ 
ring  FOT. 

-  Schedule  only  MTT  deemed  critical  to  COCOM  TSCP  priorities  during  peak  months 
and  delay  other  MTT  to  nonpeak  months. 

•  Improve  utilization  of  existing  instructors  by  restricting  leave  during  peak  season. 

•  Create  “breathing  space”  within  existing  supply. 

-  Train  U.S.  students  and  IMSs  at  separate  locations  to  accommodate  different  needs. 

-  Set  up  a  finishing  school  for  students  who  are  ECL-qualified/waiting  for  SET/FOT. 
Training  could  be  more  self-paced,  self-directed,  and  make  more  extensive  use  of  com¬ 
puter  resources  versus  instructors.  Training  could  be  more  focused  on  preparing  for 
upcoming  SET  or  FOT — study  of  military  jargon,  acronyms,  pace  of  speech,  accents, 


40 


We  assume  that  any  changes  to  the  requirement  processes  will  be  included  in  any  new  governance  documents. 
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etc.  Training  would  not  require  class  size  limitations  (one  large  continuing  running 
class).  Training  might  use  military  reservists  or  retirees  for  instructors. 

•  Consider  increasing  supply. 

-  Increase  classroom  capacity  to  200  rooms. 

-  Increase  instructor  capacity  to  175  instructors. 

-  Prepare  for  execution  of  additional  surge  options. 

-  IDIQ  temporary  instructor  contract  (40  additional) 

-  contracted  ELT  programs  (25  or  more  instructors  and  facilities  using  university  pro¬ 
grams  or  contractors) 

-  temporary  rental  facilities  and  contracted  temporary  instructors  (temporary  during 
building  phases). 


CHAPTER  FOUR 


Financial  Management  and  Cost  Recovery 


This  chapter  reviews  HQAETC  and  DLIELC’s  financial  management,  focusing  on  the  deter¬ 
mination  of  the  unit  cost  of  training.  It  describes  our  analysis  of  DLIELC  and  HQ  AETC’s 
financial  management  structures  and  procedures. 

DLIELC  appears  to  be  in  conformance  with  the  provisions  of  the  DSCA  Security  Assis¬ 
tance  Manual,  as  implemented  through  various  levels  of  regulations.  For  instance,  the  Air 
Force  establishes  tuition  rates  according  to  tuition  pricing  guidelines  for  training  interna¬ 
tional  students  as  pursuant  to  DoD  7000. 14-R,  Volume  15.  Accordingly,  rates  are  published 
by  MASL;  however,  support  items  (subsistence,  medical,  and  housing  service  charges)  are  not 
included  in  these  prices.  Moreover,  the  Air  Force  bills  other  military  departments  for  tuition 
when  international  students  enter  training  under  their  sponsorship.  Like  most  other  security 
assistance  providers,  DLIELC  also  has  established  procedures  that  govern  cost  recovery  should 
students  not  appear  for  a  programmed  course.  AFJI  16-105,  paragraph  3-7,  provides  guide¬ 
lines  for  forfeiture  charges  for  IMSs  at  DLIELC  (and  is  discussed  in  Chapter  Two). 

However,  based  on  our  analysis,  we  find  that  current  financial  management  practices 
have  not  been  sufficiently  established,  which  may  expose  ELT  clients  (which  we  define  as 
the  agencies  paying  the  costs  of  individual  ELT,  i.e.,  DoD,  Department  of  State,  and  foreign 
countries)  to  additional  financial  risk.  More  specifically,  we  conclude  that  DLIELC  is  not  in  a 
position  to  explicate  its  methodology  for  determining  costs  and  does  not  have  cost  models  that 
could  be  used  to  determine  all  the  costs  associated  with  student  training.  This  is  an  important 
financial  management  issue  because  Title  22  stipulates  that  certain  government-incurred  costs 
are  to  be  recovered.  This  chapter  proposes  a  set  of  analytical  solutions  and  makes  five  recom¬ 
mendations  to  implement  them. 


HQ  AETC  Financial  Management 

AFJI  16-103  establishes  HQAETC  responsibilities,  which  include 

•  managing,  operating,  funding  and  providing  personnel  for  DLIELC 

•  identifying  to  the  EA  all  annual  budget  and  manpower  resources  in  its  operation  and 
maintenance  budget  and  financial  plans 

•  describing  the  DLIELC  commandant’s  responsibilities,  which  include 

-  sending  an  annual  budget  and  financial  plan  for  DLIELC  to  HQAETC,  according  to 
HQAETC  and  HQU.S.Air  Force  (USAF)  direction. 
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-  preparing  and  submitting  future-years  plan  through  command  channels  to  the  Assis¬ 
tant  Secretary  of  Defense  for  Force  Management  Policy  (ASD  (FM&P)). 

The  commandant’s  plan  should 

•  identify  all  planned  ELT  needs,  approved  training  development  needs,  and  major  train¬ 
ing  support  needs 

•  identify  funds  and  personnel  needed  to  accomplish  DLIELC’s  mission  and  send  the 
requirements  through  command  channels  to  ASD  (FM&P) 

•  address  the  budget  year,  plan  year,  and  out-years  in  the  POM  submission. 

Ostensibly,  the  above  regulations  and  procedures  should  provide  an  adequate  normative 
regime  whereby  all  costs  associated  with  English  language  training  should  be  identified,  veri¬ 
fied,  and  billed  promptly  to  ELT  clients.  However,  we  found  several  unmet  needs:  (1)  the  need 
to  develop  more  comprehensive  cost  models,  (2)  the  need  to  develop  and  document  validated, 
data-based  cost  determination  procedures  to  calculate  the  full,  recoverable  costs  (as  stipulated 
in  law  and  regulations)  incurred  by  both  DLIELC  and  AETC  in  the  execution  and  manage¬ 
ment  of  ELT  to  all  students  (U.S.  and  foreign),  and  (3)  the  need  to  establish  management  pro¬ 
cedures  to  ensure  that  all  eligible  costs  expended  in  the  execution  of  ELT  are  recovered  by  the 
U.S.  Air  Force. 

Even  where  HQ  AETC  has  attempted  to  exercise  proper  oversight  of  DLIELC,  its 
financial  management  office  appears  to  be  relying  on  dated  averaged  data.  Based  on  the  lack 
of  agreed-upon  cost  models  used  by  DLIELC’s  financial  management  directorate  and  HQ 
AETC’s  acceptance  of  this  situation,  we  conclude  that  HQ  AETC  and  the  37th  Training 
Wing  do  not  have  the  tools  for  the  necessary  financial  oversight  of  DLIELC. 

HQ  AETC  leadership  and  DLIELC  management  face  a  unique  challenge  in  this  regard. 
DLIELC  is  a  unique  DoD  asset  that  provides  formal  and  regular  training  within  DoD’s  Secu¬ 
rity  Cooperation  and  Security  Assistance  structures  that  is  not  on  a  marginal  cost  basis.  In 
other  words,  with  the  exception  of  ECHO  Company,  DLIELC  does  not  premise  the  financial 
basis  of  ELT  to  foreign  nations  on  a  marginal  cost  basis  in  relation  to  the  already  ongoing 
training  for  the  U.S.  armed  forces.  Thus,  its  financial  management,  and  particularly  its  cost- 
recovery  practices,  must  be  carefully  designed  to  capture  all  financial  costs  (variable,  as  well 
as  fixed)  associated  with  the  execution  of  its  mission.  It  should  be  reiterated  that  in  the  case  of 
providing  FMS,  all  costs  associated  with  the  purchased  training,  including  uniform  military 
personnel  accelerated  salary  costs,  must  be  recovered  from  the  country  purchasing  the  train¬ 
ing.  This  should  be  included  in  a  cost  model,  as  well  as  all  incremental  costs  associated  with 
the  37th  Training  Wing  and  HQ  AETC’s  overall  management  and  financial  management  of 
DLIELC. 


Case  Study  Observations:  Financial  Management 

In  the  U.S.  security  cooperation  community,  other  organizations  are  careful  to  bill — fully  and 
accurately — all  costs  associated  with  the  contracting  client  to  the  degree  possible.  The  Center 
for  Civil-Military  Relations  (CCMR)  and  DHLS  factor  fixed  and  variable  costs  into  their  cost 
structure  for  their  courses  and  consulting  projects  undertaken  using  Security  Cooperation 
and  Security  Assistance  funds.  We  also  conducted  research  into  two  foreign  defense  organiza- 
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tions  that  execute  ELT  but  use  different  management  and  organizational  approaches:  the  UK 
Defence  School  of  Languages/English  Language  Wing,  and  the  Canadian  Forces  English/ 
French  Language  Training  Program.  Our  research  found  that  both  programs  are  careful  to 
ensure  that  the  execution  of  their  ELT  missions  are  based  on  full  cost  recovery  or  are  moving 
quickly  in  that  direction.  Infrastructure  costs  are  amortized  and  factored  into  the  organiza¬ 
tion’s  cost  model.  These  costs  (fixed  and  variable)  are  billed  to  the  requesting  client.  This  differs 
from  DLIELC,  where  infrastructure  costs  are  not  billed  (by  law,  infrastructure  costs  cannot 
be  amortized). 

Additionally,  DLIEC  maintains  a  cadre  of  senior  English  language  instructors  whose 
principal  tasks  are  curriculum  development.  This  fixed  cost  is  not  billed  either  and  is  an 
instance  of  DLIELC  exposing  its  clients  to  financial  risks  because  the  costs  of  these  individu¬ 
als  are  met  either  through  the  U.S.  Air  Force  back  to  the  DoD  or  as  “rent”  that  is  covered  by 
attending  students  in  the  form  of  higher  unit  instructional  costs.  In  either  case,  it  is  not  clear 
that  curriculum  development  is  best  done  in-house,  since  English  language  training  materials 
are  abundant  in  the  private  sector.  There  may  be  a  case  where  some  SET  material  is  best  pro¬ 
duced  in-house,  but  that  question  was  not  researched. 


Discerning  Unit  Cost 

Interviews  and  subsequent  correspondence  with  DLIELC  and  HQAETC  have  reinforced  the 
finding  that  the  37th  Training  Wing  and  HQAETC  cannot  determine  that  all  reimbursable 
costs  are  recovered.  Consequently,  DLIELC  cannot  provide  the  Department  of  the  Air  Force 
with  the  discrete  “unit  cost”  of  its  English  language  instruction.  Unit  cost  can  be  defined  in 
a  number  of  different  ways  (total  average  cost  per  hour  of  instruction,  total  average  cost  to 
advance  a  student  from  one  level  to  another,  average  fixed  cost  of  an  MTT  or  LTD,  etc.). 

Without  such  data  on  unit  costs,  in  addition  to  not  being  able  to  recover  the  costs,  the 
Air  Force,  OSD,  DSCA,  and  the  Department  of  State  are  unable  to  determine  the  cost- 
effectiveness  of  DLIELC’s  English  language  instruction.  Moreover,  without  actual  unit  costs, 
the  government  cannot  go  to  the  “market”  to  conduct  cost  comparisons  through  price  discov¬ 
ery.  With  full  knowledge  of  the  various  unit  costs  associated  with  DLIELC’s  English  language 
activities,  DoD  would  be  in  an  extremely  strong  position  to  redesign  the  structure  and  pro¬ 
cedures  of  the  institution  in  accordance  with  comparative  costs.  Such  information  could  also 
enable  DoD  to  change  the  management  of  ELT  by  finding  other,  more  cost-effective  providers 
in  certain  instructional  or  support  activities. 


Proposed  Solutions 

The  study  sponsors  requested  that,  as  part  of  the  study’s  directive  to  assist  DLIELC  in  its  stra¬ 
tegic  planning,  we  develop  costing  models.  RAND  developed  the  following  models: 

•  Phase  1:  Direct  in-processing  costs 

•  Phase  2:  Direct  fixed  costs 

•  Phase  3:  Direct  variable  costs 


36  Optimizing  the  Defense  Language  Institute  English  Language  Center 


•  Phase  4:  Direct  out-processing  costs 

•  Phase  5:  Indirect  overhead  costs. 

The  models  were  developed  in  Excel.  For  reference,  they  appear  in  Appendix  B,  with 
detailed  explanatory  notes  to  assist  DLIELC  financial  management  in  initiating  their 
implementation. 

Once  management  is  convinced  that  the  models  produce  costing  data  within  acceptable 
parameters  of  margin  of  error,  the  models  should  be  used  to  develop  a  cost  library  to  inform 
decisionmaking.  Over  time,  the  models  can  be  adjusted  with  experience  so  that  their  accuracy 
and  fidelity  of  data  improve. 


Recommendations 

Since  DLIELC,  the  37th  Training  Wing,  and  HQAETC  have  not  collaborated  on  the  devel¬ 
opment  of  adequate  costing  tools,  there  exists  a  need  for  a  systemic  review  of  existing  manage¬ 
ment  procedures  to  ensure  that  cost  recovery  of  ELT  is  an  essential  component  of  its  manage¬ 
ment  practices. 

We  also  recommend  the  following  actions: 

•  Form  a  task  force  of  senior  leadership  from  37th  Training  Wing,  HQ  AETC,  and 
DLIELC  with  the  objective  of  implementing  cost  recovery  reform. 

•  Have  the  task  force  review  proposed  costing  models  to  ensure  their  methodological  accu¬ 
racy.  This  review  should  be  informed  by  a  comprehensive  review  of  U.S.  Security  Assis¬ 
tance  regulations  to  ascertain  how  much  USAF  management  and  oversight  costs  are  fully 
captured  in  the  models,  as  stipulated  in  U.S.  Code  Title  10,  and  particularly  Title  22. 
Care  needs  to  be  exercised  in  the  review  of  the  cost  models  to  ensure  that  there  is  a  clear 
methodological  relationship  between  DLIELC’s  costing  methodology  and  HQAETC’s 
family  of  cost  models. 

•  Once  the  models’  methodologies  have  been  validated  and  agreed  upon  among  all  three 
parties,  fully  cost  DLIELC’s  activities.  The  resulting  data  need  to  be  fully  vetted  by  all 
three  organizations;  once  agreement  is  reached,  a  DLIELC-specific  costing  library  can  be 
developed  and  managed,  in  accordance  with  USAF  policy  and  AETC  regulations,  and 
used  by  all  three  parties  as  an  essential  element  of  DLIELC’s  financial  management. 

•  After  the  costing  library  is  developed,  review  and  re-issue  financial  management  policies 
and  procedures  as  they  relate  to  DLIELC. 


CHAPTER  FIVE 


Technology 


As  noted  in  Chapter  Two,  the  Air  Force  values  the  smart  use  of  available  technology  in  instruc¬ 
tional  systems.  This  chapter  reviews  DLIELC’s  use  of  technology  and  contrasts  it  with  obser¬ 
vations  on  blended  learning  (a  combination  of  classroom-  and  computer-based  training)  and 
distance  learning. 

Based  on  our  analysis  we  find  that,  to  date,  DLIELC  has  not  been  introducing  new  tech¬ 
nologies  at  a  quick  enough  rate,  despite  the  fact  that  their  successful  use  could  lower  fixed  and 
variable  costs.  More  specifically,  we  were  unable  to  locate  a  technology  roadmap  for  DLIELC.1 
While  we  do  not  believe  that  DLIELC  needs  to  be  an  early  adopter  of  technology,  there 
are  proven  educational  technologies  that  could  enhance  training  and  possibly  decrease  DoD’s 
costs.  This  chapter  elaborates  on  this  Ending  and  concludes  with  a  set  of  five  recommendations. 


DLlELC's  Use  of  Technology 

Because  new  weapon  systems  and  technology  are  rapidly  being  developed,  curriculum  devel¬ 
opers  must  ensure  that  ELT  materials  are  regularly  updated.  However,  changes  in  technology 
will  not  uniformly  affect  curriculum  in  SET  and  GET.  True  to  its  namesake,  the  SET  cur¬ 
riculum  is  specialized  and  will  be  more  sensitive  to  changes  in  military  technology.  Unfortu¬ 
nately,  the  time  to  develop  new  curricula  at  DLIELC  is  quite  protracted.  Estimates  provided 
by  DLIELC  indicate  that  it  takes  approximately  2,500  hours  to  develop  one  week  of  SET  cur¬ 
riculum.2  Based  on  these  estimates,  it  would  be  difficult  for  curriculum  development  to  keep 
pace  with  changes  in  specialized  terminology. 

Other  specialty  fields  have  also  recognized  the  difficulty  of  maintaining  pace  with  advanc¬ 
ing  educational  methods  (most  often  associated  with  technology  solutions).  To  respond  to 
these  challenges,  some  researchers  have  advocated  integrating  curricula  and  technology.  For 
example,  textbooks  using  collaborative  platforms  (e.g.,  Wikis)  have  provided  one  solution  for 
quickly  updating  material  without  having  to  create  a  new  edition  for  publication.  Because 
DLIELC  does  not  sell  SET  textbooks,  combining  a  collaborative  platform  and  digital  textbook 
technology  would  allow  subject  matter  experts  at  FOT  sites  to  identify  and  update  outdated 


1  The  real  value  of  a  technology  roadmap  is  the  process  and  the  resulting  identification  of  critical  technology  investment 
decisions  (Marie  L.  Garcia  and  Olin  H.  Bray,  “Fundamentals  of  Technology  Roadmapping,”  Albuquerque,  N.M.:  Sandia 
National  Laboratories,  1997). 

2  According  to  feedback  from  DLIELC,  “[f]or  [curriculum  development]  to  produce  1  module  (one  week  of  training)  [it] 
takes  a  minimum  of 2500  hours.  This  includes  the  development  of  the  text,  instructor  guide,  ancillary  materials,  and  assess¬ 
ment.  Producing  materials  that  have  electronic  media  components  take  at  a  minimum  twice  that  amount  of  time.” 
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terminology.  In  addition,  updates  could  be  made  quickly  to  incorporate  other  stakeholders, 
including  DLIELC  instructors,  students,  and  test  developers  who  provide  feedback  about  the 
curriculum. 

DLIELC  has  indicated  that  it  is  using  technology  more  in  the  classrooms  (e.g.,  websites) 
but  that  not  all  students  have  access  to  the  Internet.  Currently,  only  certain  classes  are  assigned 
to  the  Learning  Center  where  Internet  access  is  provided.  Limited  access  to  computers  and  the 
Internet  may  impede  the  introduction  of  potentially  beneficial  technology. 

Although  testing  for  skills  has  incorporated  additional  assessments  for  listening,  there 
are  opportunities  to  expand  the  use  of  the  OPI.  The  OPI  is  required  only  for  certain  students 
and  requires  considerable  training  and  resources  on  the  part  of  DLIELC  staff.  These  resource 
demands  restrict  practice  opportunities  for  students  who  need  to  pass  the  OPI.  Also,  other  stu¬ 
dents  learning  English  may  benefit  from  practicing  listening  and  speaking. 

Recently,  the  American  Council  on  the  Teaching  of  foreign  Languages  developed  a  com¬ 
puterized  version  of  the  OPI  (OPIc®).  The  OPIc®has  shown  acceptable  reliability  and  validity3 
and  may  provide  students  who  need  to  pass  the  OPI  with  additional  opportunities  to  practice, 
furthermore,  the  OPIc®  can  be  group-administered,  allowing  many  students  to  take  the  test 
online  at  the  same  time.  Introducing  this  technology  could  further  provide  opportunities  for 
independently  evaluating  English-speaking  skills  for  those  students  not  required  to  take  the 
OPI  for  POT.  Taking  advantage  of  most  of  the  technological  solutions  will  require  issuing  lap¬ 
tops  to  a  larger  percentage  of  students. 


The  Blended  Learning  Technology  Approach 

figure  5.1  provides  an  illustration  of  a  potential  technology  usage  that  could  save  money  but 
that  has  not  been  tested.  On  the  left-hand  side  of  the  figure  is  the  current  DLIELC  approach 
of  five  to  six  hours  of  classroom  instruction  in  a  nine-hour  day.  The  current  approach  leaves 
all  classrooms  at  75  percent  maximum  utilization,  meaning  that  there  is  unused  capacity  that 
could  accommodate  about  300  more  students. 

If  it  was  possible  to  substitute  some  of  the  classroom  instruction,  possibly  two  hours, 
with  computer-based  training  (utilizing  the  current  computer  labs,  library,  or  dorms/quarters), 
DLIELC  could  double  its  capacity  from  900  students  to  1,800  students  with  no  change  in 
the  duty  day.  In  this  example,  the  1,800  students  would  be  divided  into  two  groups,  with  one 
using  the  classrooms  while  the  other  gets  computer-based  training;  after  lunch,  the  group  that 
got  classroom  instruction  in  the  morning  would  get  computer-based  training  in  the  afternoon 
so  that  the  group  that  had  computer-based  training  in  the  morning  could  make  use  of  the 
classrooms  in  the  afternoon. 

The  idea  suggested  in  figure  5.1  has  not  been  tested  for  effectiveness.  We  recommend  a 
small-scale  pilot  test  to  determine  the  effectiveness  of  the  idea.  This  test  would  require  that  all 
students  have  laptops,  both  those  in  the  control  and  those  in  the  test  group. 


3  E.  A.  Surface,  R.  M.  Poncheri,  and  K.  S.  Bhavsar,  Two  Studies  Investigating  the  Reliability  and  Validity  of  the  English 
ACTFL  OPIc®  with  Korean  Test  Takers:  The  ACTFL  OPIc®  Validation  Project  Technical  Report,  Raleigh,  N.C.:  SWA  Con¬ 
sulting,  2008. 
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Figure  5.1 

Example  of  a  Blended  Learning  Technology  Approach 


9-hour  day 


Current  Group  A  Group  B 


1,800 

students 

Instructor  day: 

7  hours  plus  prep 


*Theoretical  limit  given  150  classrooms  and  6:1  student-to-teacher  ratio. 

RAND  77)7(52-5.) 


Case  Study  Observations:  Distance  Learning 

A  key  observation  in  the  area  of  technology  innovation  is  that  DLIELC  has  not  adopted  for¬ 
eign  language  technical  support  capabilities,  which  might  reduce  DLIELC’s  operating  costs. 
We  did  not  see  much  evidence  that  DLIELC  has  pursued  innovative,  commercially  developed 
technological  solutions  to  ELT.  In  contrast,  for  example,  its  sister  unit,  the  Defense  Language 
Institute  Foreign  Language  Center  (DLIFLC),  has  pursued  on-line  instruction  of  alumni. 

Several  organizations  we  examined  also  engage  in  distance  learning  practices.  The  Defense 
Institute  for  Security  Assistance  Management  (DISAM)  is  one  of  the  best  examples  to  high¬ 
light.  Two  of  the  four  main  security  cooperation  courses — the  more  junior-level  courses — are 
conducted  online.  DISAM  spent  a  year  developing  a  database  of  DoD  positions  to  identify 
staff  in  need  of  some  kind  of  security  cooperation/security  assistance-related  training.  DISAM 
identified  11,000  relevant  DoD  positions,  which  has  helped  it  target  its  course  training  options. 

Another  important  lesson  comes  from  the  Canadian  English/French  language  training 
program  for  the  Canadian  Forces,  which  has  used  commercial,  off-the-shelf  English/French 
language  distance  learning  programs  for  students  to  learn  the  other  official  language  of  Canada. 
This  has  been  found  to  be  content-effective  in  terms  of  learning  foreign  languages,  as  well  as 
cost-effective. 


Recommendations 

As  noted  above,  based  on  our  analysis,  we  find  that,  to  date,  DLIELC  has  not  been  introducing 
new  technologies  at  a  quick  enough  rate,  despite  the  fact  that  their  successful  use  could  lower 
fixed  and  variable  costs.  The  RAND  team  suggests  that  DLIELC  start  with  an  interdisciplin- 
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ary  team  made  of  up  leadership,  technologists,  instructors,  and  infrastructure  experts  and  then 
develop  a  vision  of  what  cost-effective  technology  can  do  for  DLIELC.  From  there,  it  would 
run  small  pilot  programs  to  test  various  technological  solutions,  using  the  Horizon  reports4  as  a 
starting  point.  DLIELC  should  maintain  a  database  of  lessons  learned  from  the  pilot  programs 
to  refine  the  requirement.  It  should  also  involve  instructors  in  the  process  so  that  the  efforts  are 
not  top -down- directed  but  bottom-up-experienced.  Finally,  it  should  select  full-scale  programs 
for  implementation  when  the  vision  is  met  and  tests  are  cost-effective. 

Given  what  we  have  found,  we  believe  that  DLIELC  could  use  technology  more  effec¬ 
tively  and  should  explore  all  possible  means  of  using  new  and  existing  cost-effective  ELT  tech¬ 
nological  support  tools  and  means,  including  the  following: 

•  Explore  where  there  may  be  cost-effective  means  to  deliver  some  ELT  through  distance 
learning.5  There  are  justifiable  reasons  why  partner  countries  require  in-person  instruc¬ 
tion,  but  in  some  cases  DLIELC  needs  to  consider  web-based  ELT,  if  not  to  replace,  then 
at  least  to  augment  in-person  training  through  a  blended  learning  technology  approach. 

•  Explore  computerized  methods  such  as  OPIc®. 

•  Consider  using  a  collaborative  platform  approach  to  update  SET  textbooks. 

•  Provide  all  students  (not  just  SET  students)  with  laptops,  or  use  laptops  as  an  incentive 
to  reach  some  level. 

•  Buy  commercial  bandwidth  needed  for  Internet  usage  and  add  those  costs  to  the  fixed 
cost  of  running  the  schoolhouse. 

If  implemented  as  part  of  a  comprehensive  roadmap,  these  recommendations  will  ensure 
that  DLIELC  is  in  a  position  to  leverage  current  and  emerging  technologies  to  enhance  its 
training  delivery.  Such  a  roadmap  will  serve  as  a  useful  tool  in  overcoming  other  challenges 
and  free  up  manpower  in  the  classroom  and  create  greater  efficiencies  in  study  options  for  stu¬ 
dents,  curriculum  development,  and  training  materials  review. 


4  The  Horizon  reports  describe  the  latest  in  educational  technologies  with  estimated  horizons  until  maturity.  The  reports 
are  put  together  by  a  consortium  of  technologists  and  educational  experts. 

5  For  instance,  all  partner  countries  using  MTTs  may  not  need  physical  ELT  instruction. 


CHAPTER  SIX 


Identity 


DLIELC  fulfills  three  broad  roles.  First,  as  a  training  institution,  it  provides  students  with  high- 
quality  ELT  that  will  allow  them  to  meet  the  language  requirements  of  their  FOT.  Second, 
as  a  military  institution,  it  is  a  first  step  for  U.S.  and  international  trainees  toward  receiving 
military  education  in  the  United  States.  And  third,  as  a  security  cooperation  organization,  it 
provides  U.S.  partner  countries  with  a  venue  where  IMSs  can  learn  about  American  culture, 
develop  relationships  with  their  U.S.  counterparts,  and  improve  their  country’s  armed  forces’ 
interoperability  with  the  United  States.  These  three  roles  are  critical  to  DLIELC’s  mission,  yet 
they  create  tensions  at  times. 

This  chapter  examines  these  three  roles,  focusing  on  the  tensions  among  them.  Based 
on  the  results  of  a  set  of  case  studies  that  examine  similar  training  organizations,  we  find 
that  DLIELC’s  conflicting  priorities  have  led  to  contradictory  views  on  the  institutions  identity 
(i.e.,  what  is  its  primary  purpose — an  academic,  military,  or  government/policy  institution l).  As  a 
result,  DLIELC’s  institutional  priorities  appear  vague,  as  is  its  definition  of  what  would  con¬ 
stitute  achieving  “success.”  We  conclude  with  a  set  of  seven  recommendations  for  clarifying 
DLIELC’s  identity.  If  implemented,  these  recommendations  will  permit  the  organization  to 
move  forward  with  the  proper  focus. 


Critical  Function  Played  by  DLIELC 

DLIELC’s  purpose  is  to  teach  English  to  international  and  U.S.  students.  The  Air  Force  Cul¬ 
ture,  Region,  &  Language  Flight  Plan  (2009)  states  that  “[t]he  end-state  sought  will .  .  .  provide 
coalition  and  partner  nations  with  the  English  language  skills  needed  to  maximize  our  abil¬ 
ity  to  operate  together.”  DLIELC  plays  an  essential  role  in  DoD’s  overall  security  cooperation 
strategy  as  the  gateway  to  FMS  and  IMET  training  and  education.  It  is  often  the  entry  point 
for  foreign  students  experiencing  U.S.  culture  directly.  DLIELC  can  also  be  the  first  step  of 
a  longer  relationship  between  international  trainees  and  the  U.S.  staff,  especially  for  students 
who  return  for  additional  courses.  The  type  of  relationships  that  DLIELC  builds  through  ELT 
has  been  recognized  as  highly  valuable.  In  fact,  the  U.S.  Government  Accountability  Office 
(2010)  recalled  that  “the  Under  Secretary  for  Public  Diplomacy  and  Public  Affairs  has  identi¬ 
fied  English  language  teaching  as  the  single  most  powerful  public  diplomacy  tool  available  to 
public  affairs  officers”  (p.  20).  Earlier  in  2009,  the  Congressional  Research  Service  warned  that 
“reduction  in  such  language  instruction  represents  lost  opportunity  to  engage  foreign  pub¬ 
lics  and  to  encourage  long-lasting  connections  with  and  goodwill  toward  the  United  States” 

(p.  48). 
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To  fulfill  its  building  partnership  (BP)  mandate,  DLIELC  organizes  cultural  activities  as 
part  of  the  DoD  Field  Studies  Program  to  familiarize  IMSs  with  American  institutions  and 
values.  The  stated  purpose  of  this  program  is  to 

ensure  that  international  students  return  to  their  homelands  with  an  understanding  of  the 
responsibility  of  governments,  military  and  citizens  to  protect,  preserve  and  respect  the 
rights  of  every  individual.1 

DLIELC  also  put  in  place  a  sponsorship  program — American  Members  of  International 
Goodwill  to  Others  (AMIGO) — through  which  students  are  able  to  interact  on  a  personal 
basis  with  individuals  and  families  in  the  San  Antonio  area  to  get  a  first-hand  experience  of  the 
American  way  of  life. 


DLlELC's  Training  Role 

DLIELC  Fulfills  its  training  (or  academic)  objective  reasonably  well.  The  school  is  accredited 
by  the  Commission  on  English  Language  Program  Accreditation  (CEA),  which  is  the  standard 
in  English  as  a  second  language  (ESL)  teaching.  It  received,  upon  inspection,  “outstanding” 
and  “excellent”  evaluations.  DLIELC  instructors  have  strong  training  backgrounds  in  ESL/ 
English  as  a  foreign  language  (EFL),  and  most  have  a  master’s  degree.  An  overwhelming  major¬ 
ity  of  students  report  being  satisfied  with  their  linguistic  progress.  In  2009,  88  percent  of  GET 
students  “strongly  agreed”  or  “agreed”  with  the  statement  that  their  English  listening  skills  had 
improved  during  their  time  at  DLIELC.  Eighty-four  percent  provided  a  similar  response  for 
reading  skills,  83  percent  for  speaking  skills,  and  71  percent  for  writing  skills.  Over  the  years, 
the  percentage  of  IMSs  who  reach  their  ECL  target  score  has  remained  relatively  stable  and 
high  (91  percent  on  average).2  The  number  of  U.S.  students  who  fail  to  ECL-qualify  experi¬ 
enced  a  dramatic  surge  from  FY03  to  FY06  before  decreasing  from  approximately  70  percent 
in  FY06  to  13  percent  in  FY09  (Figure  6.1). 

Still,  DLIELC  as  a  training  institution  has  experienced  challenges.  A  large  number  of 
students  see  their  training  cut  short  before  they  ECL-qualify:  87.3  percent  of  U.S.  students  and 
28.2  percent  of  IMSs  who  did  not  ECL-qualify  received  a  curtailment  in  2005-2010.  Some 
students  received  a  waiver  to  move  on  to  their  planned  FOT,  even  when  they  did  not  reach 
the  required  ECL  score.  It  is  important  to  note,  however,  that  this  generally  happens  with 
students  who  are  very  close  to  their  target  ECL  and  would  likely  miss  the  beginning  of  their 
FOT  if  they  remained  at  DLIELC  for  the  few  additional  weeks  of  ELT  that  they  need.  It  also 
applies  to  a  very  small  number  of  students:  Only  1.3  percent  of  IMSs,  on  average,  received  an 
ECL  waiver  in  2005-2009.3  Still,  it  suggests  that  the  ECL  qualification  criteria  to  move  on  to 
FOT  have  limited  validity  and  that  the  “wheel,”  which  assigns  12  weeks  of  English  training  to 
students  with  an  initial  ECL  score  of  85-89  and  who  need  a  90,  may  have  limited  usefulness 
as  a  programming  tool.  There  is  also  insufficient  evidence  about  the  validity  of  ECL  scores  as 


1  DLIELC  website,  “Field  Studies  (LEF),”  undated. 

2  The  annual  percentage  of  IMSs  reaching  their  target  ECL  score  ranged  from  86  to  94  percent  in  FY01-FY09. 

3  The  annual  percentage  of  IMSs  receiving  an  ECL  waiver  ranged  between  0.5  and  3  percent  in  2005-2009. 
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Figure  6.1 

Percentage  of  IMSs  and  U.S.  Students  Who  Did  Not  ECL-Qualify,  FY01-FY09 


RAND  TR11S2  6.1 


a  metric  of  English  proficiency.4  A  sample  of  data  provided  to  the  research  team  by  DLIELC 
indicates  a  relationship  between  ECL  and  the  Test  of  English  as  a  Foreign  Language  (TOEFL), 
but  this  sample  represents  only  a  very  small  subset  of  the  students  who  attend  DLIELC.  The 
2009  Annual  Trend  Report  for  Direct  Mail  Questionnaires  (DMQs)5  concluded  rather  alarm¬ 
ingly  that  “[t]he  OPI/ECL  scores  were  not  reliable  indicators  of  success;  neither  were  SET 
assessments.”  Additional  issues  include  the  fact  that  DLIELC  students  do  not  self-assess  for 
placement,  progression,  or  curriculum,  although  such  procedures  are  commonly  considered 
important  elements  of  adult  education.  Finally,  a  number  of  instructors  are  diverted  to  cur¬ 
riculum  development  activities,  creating  a  shortage  of  teaching  staff. 

Part  of  DLIELC’s  failure  may  be  a  result  of  the  instructor  staff  seeing  themselves  as  edu¬ 
cators  and  not  as  Air  Force  instructors.  In  an  education  environment,  such  as  a  college,  the 
responsibility  for  learning  is  on  the  student.  In  a  military  training  environment,  the  instructors 
have  more  of  a  responsibility  for  ensuring  that  individuals  are  trained  correctly.  The  DLIELC 
staff  should  put  less  effort  in  curriculum  development,  possibly  even  outsourcing  it,  and  more 
emphasis  on  assessment  and  improving  the  training. 


4  This  is  not  meant  to  suggest  that  the  ELT  itself  provided  by  DLIELC  is  not  valid.  On  the  contrary,  student  feedback 
clearly  states  that  most  students  are  satisfied  with  their  instruction. 

'  The  Annual  Trend  Report  is  a  summary  and  analysis  of  the  answers  given  by  former  DLIELC  students  and  their  FOT 
instructors  to  the  direct  mail  questionnaire  (DMQ).  It  is  compiled  every  calendar  year  by  the  Standardization  and  Evalua¬ 
tions  Division  in  DLIELC. 
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DLlELC's  Military  Role 

IMSs’  English  language  skills,  for  those  who  graduate,  are  generally  sufficient  to  successfully 
attend  FOT,  as  evidenced  by  their  providing  an  overall  rating  of  “satisfactory”  from  DMQs 
in  2009.  Here  too,  however,  DLIELC  faces  a  number  of  challenges.  They  include  the  delays 
experienced  by  some  students  between  the  end  of  their  English  language  training  and  the  start 
of  their  FOT,  or  the  difficulty  of  some  students  in  obtaining  the  medical  clearance  they  need 
for  their  FOT.6  Another  issue  is  the  lack  of  preparation  of  IMSs  for  the  military  environment 
they  will  experience  during  their  FOT.  DFIEFC’s  environment  for  IMSs  is  almost  exclusively 
civilian,  and  IMSs  do  not  undergo  the  same  drills  or  physical  training  (PT)  requirements  as 
U.S.  students  do.  DFIEFC’s  requirements  in  terms  of  discipline,  too,  are  very  lax  compared  to 
what  will  be  expected  of  IMSs  during  their  FOT — or  of  what  is  expected  of  U.S.  students  at 
DFIEFC.  The  school’s  limited  ability  to  enforce  discipline  with  IMSs  represents  a  major  chal¬ 
lenge  for  the  teaching  and  administrative  staff.7 

A  related  concern  is  the  fact  that  English  language  skills  taught  to  IMSs  are  not  always 
targeted  toward  their  FOT.  Field  evaluation  visits,  in  particular,  report  on  several  occasions  in 
which  IMSs  are  not  familiar  with  military  jargon  or  the  extensive  use  of  acronyms  in  the  U.S. 
military.  Students  themselves  acknowledge  this  shortcoming:  In  2009,  62  percent  of  respon¬ 
dents  agreed  or  strongly  agreed  with  the  statement  “I  need  to  learn  more  military  language/ 
acronyms/abbreviations  and  slang.”8  Another  issue  relates  to  IMSs’  speaking  and  writing  skills 
being  weaker  than  their  reading  and  comprehension  skills.  Yet,  the  former  skills  are  critical 
for  some  FOT.  The  most  severe  judgment  on  DFIEFC’s  ability  to  fulfill  its  military  role  came 
from  the  school  itself,  which  stated  in  the  conclusion  of  its  2009  Annual  Trend  Report  for 
Direct  Mail  Questionnaires: 

Even  though  the  overall  DMQ  “trend”  was  satisfactory,  the  reasons  for  a  satisfactory  trend 

could  not  be  determined.  .  .  .  Qualitatively  and  quantitatively,  it  was  unclear  as  to  the 

actual  impact  of  DLIELC  training  on  graduates’  FOT  success. 


DLlELC's  Security  Cooperation  Role 

The  Air  Force  has  recently  called  “building  partnerships”  one  of  its  core  functions.  BP  is  a 
subset  of  security  cooperation,  and  is  the  only  organization  in  the  AF  that  is  strictly  focused 
on  that  function.  With  regard  to  security  cooperation,  several  elements  of  success  suggest  that 
DLIELC  fulfills  this  role  relatively  well.  DLIELC  currently  meets  all  demands  from  services 
for  resident  and  nonresident  training.  DMQs,  end-of-course  (EOC)  surveys,  and  reports  from 


6  Delays  in  obtaining  medical  clearances  can  however  hardly  be  imputed  to  DLIELC  and  are  largely  related  to  the  quality 
of  medical  care  in  students’  countries  of  origin. 

7  During  a  visit  of  the  research  team  to  San  Antonio,  DLIELC  personnel  mentioned  that  there  are  about  50  students  who 
“are  not  really  there”  for  training,  and  added  that  people  who  lose  their  motivation  are  a  drag  on  the  classroom.  Yet,  disci¬ 
plinary  action  against  IMSs  is  extremely  rare  at  DLIELC. 

8  Only  respondents  who  gave  a  neutral  or  negative  answer  to  three  previous  questions  on  language  skills  were  asked  this 
question.  Hence,  they  represent  62  percent  of  a  sample  equivalent  to  approximately  one-quarter  of  the  initial  sample  of  SET 
respondents. 
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field  trips  show  that  IMSs  generally  enjoy  their  experience  at  DLIELC,  whether  in  the  class¬ 
room  or  during  field  visits. 

A  large  majority  of  IMSs  are  officers  who  may  eventually  come  to  occupy  a  position  of 
influence  in  their  country  of  origin.  In  FY06-09,  75  percent  of  all  IMSs  were  officers  (an 
additional  2  percent  were  warrant  officers);  19  percent  were  enlisted.  As  shown  in  Figure  6.2, 
almost  75  percent  of  these  officers  were  in  the  0-1  to  0-3  pay  grade  range  (57  percent  of  all 
military  personnel),  with  the  remaining  25  percent  being  0-4  to  0-6. 

Seven  officers  had  a  rank  of  0-7  and  higher  in  2005,  12  in  2006,  10  in  2007,  7  in  2008, 
10  in  2009,  and  5  in  2010.9  This  figure  underlines  the  importance  of  DFIEFC  as  a  venue  to 
establish  relations  with  foreign  military  personnel.  One  might  expect  DFIEFC  would  encour¬ 
age  the  development  of  a  professional  network  among  its  alumni,  or  at  least  maintain  con¬ 
tact  with  its  graduates  to  advance  U.S.  interest  in  fostering  long-term  relationships  with  these 
individuals  and  their  institutions  and  governments  after  they  return  to  their  homeland.  This 
should  be  considered  an  important  missed  opportunity. 

The  security  cooperation  objective  of  DFIEFC  is  also  challenged  by  the  fact  that  the 
budget  provided  by  AFSAT  for  cultural  activities  has  not  changed  in  the  past  three  years  and 
is  not  planned  to  increase  any  time  soon,  while  the  number  of  students  has  grown  and  will 
probably  continue  to  grow.  According  to  regulation10  the  cost  of  the  field  studies  program  is 
included  in  the  cost  of  tuition.* 11  It  is  unclear,  however,  what  value  or  percentage  of  tuition  this 
represents.  Thirty-five  dollars  per  student  per  week  is  a  commonly  accepted  value  but  it  does 


Figure  6.2 

IMS  Pay  Grades  (Military  Only),  FY06-FY09 


2% 


RAND  TR1 152-6.2 


4  These  figures  are  as  of  early  November  2010. 

10  See  AFI  16-105  Chapter  11,  Section  III  IP  Funding,  11-6,  and  DoD  5105. 38-M,  Security  Assistance  Management  Manual, 
2003,  Section  CIO. 11,  Field  Studies  Program. 

11  Email  correspondence  with  DLIELC,  February  18,  2011. 
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not  seem  to  have  any  concrete  basis.12  In  any  case,  this  low  figure  suggests  that  field  trips  are 
given  a  low  priority. 

An  interesting  DLIELC  initiative  to  provide  IMSs  with  a  richer  cultural  experience  is  to 
pair  them  with  American  sponsors  through  the  AMIGO  program.  The  program  seems  highly 
successful:  In  2009,  79  percent  of  students  who  took  part  in  AMIGO  agreed  or  strongly  agreed 
with  the  statement:  “My  AMIGO  helped  me  improve  my  English  language  skills.”  Seventy-six 
percent  agreed  or  strongly  agreed  that  “I  learned  about  the  American  family  and  community 
life  from  my  AMIGO  sponsor.”13  However,  not  every  student  finds  a  sponsor:  out  of  the  801 
students  who  asked  for  an  American  sponsor  in  2009,  303  (38  percent)  could  not  get  one. 


The  Clash  of  Roles 

DLIELC’s  three  roles  impose  very  different  requirements  and  can  lead  to  tensions  or  conflict¬ 
ing  priorities.  For  instance,  the  academic  role  would  impose  minimum  entry  standards,  while 
the  need  to  cater  to  the  needs  of  U.S.  allies  and  partners  makes  an  “accept  all”  policy  more 
appropriate.  Similarly,  the  necessity  to  properly  program  students  in  a  training  institution 
would  warrant  a  prioritization  and/or  slot  system,  while  security  cooperation  objectives  are 
adequately  served  by  the  current  open  door  policy.14  Although  they  seem  closely  related,  the 
two  objectives  of  military  training  and  building  partnerships  also  create  tensions.  Focusing  on 
the  quality  of  military  training  would  suggest  limiting  student  load  to  ensure  adequate  billet¬ 
ing  and  base  resources.  However,  if  security  cooperation  is  an  important  goal,  then  DLIELC 
should  not  only  continue  to  accept  all  students  but  also  have  surge  capacity  to  maximize  FMS 
cases  in  the  future.  The  tension  between  the  academic  and  security  cooperation  objectives  is 
also  apparent  in  the  teaching  staff’s  hiring  criteria:  instructors  have  an  ELT  background  but  no 
policy/international/intercultural  relations  background.  DLIELC’s  policy  of  maintaining  tight 
control  over  SET  books — instead  of  disseminating  them  to  partner  countries — makes  sense 
from  a  business  point  of  view,  but  not  if  security  cooperation  is  the  main  focus. 


Case  Study  Observations:  Identity 

In  the  case  study  analysis,  we  considered  whether  other  DoD  organizations  faced  similar 
issues.  Overall,  we  found  that  organizations  closely  aligned  to  DSCA  and  OSD  guidance, 
that  report  to  DSCA  or  directly  to  OSD(P),  tend  to  be  more  government/policy-oriented  than 
those  reporting  to  one  of  the  Services. 

Organizations  such  as  DHLS,  DISAM,  CCMR,  and  COE-DMHA  are  aware  of  OSD 
and  COCOM  Theater  Campaign  Plan  guidance  and  align  their  activities  with  the  guidance 
outlined  in  those  strategic  planning  documents  for  the  most  part.  There  is  a  clear  and  logical 
rationale  for  this  adherence  to  policy  guidance  and  priorities.  The  funding  and  the  ultimate 


12  The  RAND-developed  cost  model  for  DLIELC,  Appendix  B,  Table  B.5,  seems  to  suggest  that  the  figure  is  reasonable. 

13  DLIELC,  “End-of-Course  Survey  (EOCs)  Annual  Trend  Report,  CY  2009,”  p.  22. 

14  There  are  limits  to  this  argument:  Opening  the  door  to  the  extent  that  the  quality  of  DLIELC’s  services  to  its  students 
deteriorates  would  likely  hamper  the  security  cooperation  objective. 
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long-term  viability  of  these  organizations  depend  on  providing  courses  and  consulting  projects 
that  are  in  accordance  with  policy  objectives. 

Organizations  reporting  to  a  Service,  such  as  IAAFA,  DIMO,  and  Western  Hemisphere 
Institute  for  Security  Cooperation  (WHINSEC),  are  another  step  removed  from  national-level 
policy  guidance  and  therefore  see  themselves  more  as  military  organizations.  Policy  objectives 
are  there,  but  they  appear  to  the  management  of  these  organizations  as  somewhat  opaque,  if 
not  distant. 

An  interesting  case  study  of  another  organization  that  has  an  English  language  training 
mission  is  the  E1K  Defence  School  of  Languages/English  Language  Wing  (DSL/ELW).  DSL/ 
ELW  is  unquestionably  a  defense  organization  under  the  Ministry  of  Defence’s  central  fund¬ 
ing.  It  delivers  ELT  to  fee-paying  foreign  nationals  and  has  no  pretensions  of  being  an  aca¬ 
demic  institution.  Courses  are  managed  by  a  dedicated  military  course  officer;  a  defense  civil 
servant  oversees  the  standard  of  instruction  and  also  instructs.  Equally,  the  Canadian  Forces 
English/French  Language  instructional  program  exists  solely  to  instruct  those  languages  to 
members  of  the  Canadian  Forces;  again,  it  makes  no  claim  to  being  an  academic  institution. 

What  appears  to  be  critical  from  these  case  studies  is  the  importance  of  clarity  in  mission, 
which  can  be  found  in  the  form  of  guidance  from  DSCA,  informed  by  a  careful  review  of  the 
GEF,  as  well  as  clear  lines  of  command  and  financial  control  and  expectations.  It  is  of  concern 
that  DLIELC  is  not  an  engrained  component  of  the  overall  security  cooperation  community 
within  the  Air  Force  or  at  the  Joint  level,  although  it  has  a  key  role  to  play  in  security  coopera¬ 
tion.  Indeed,  one  could  argue  that  many  of  DLIELC’s  challenges  are  compounded  by  the  fact 
that  is  does  not  firmly  belong  to  this  community.15 


Recommendations 

As  we  note  above,  we  find  that  conflicting  priorities  of  DLIELC  have  led  to  contradictory 
views  on  the  institution’s  identity  (what  is  its  primary  purpose — is  it  an  academic,  military,  or 
government/policy  institution?).  As  a  result,  DLIELC’s  institutional  priorities  appear  vague, 
as  is  its  definition  of  what  constitutes  success.  This  further  emphasizes  the  need  expressed  in 
Chapter  Two  for  overarching  guidance. 

Given  this  fact,  OSD  and  USAF  need  to  emphasize  the  government/policy-oriented 
aspects  of  DLIELC  as  an  instrument  of  the  security  cooperation  community.  Simply  stated, 
DLIELC,  like  all  security  assistance  and  security  cooperation  organizations  within  DoD,  is  an 
instrument  of  U.S.  government  policy.  All  management  assumptions  related  to  its  structure 
and  functions  must  fully  emphasize  this  principle.  Overall,  DLIELC  is  a  training  organization 
with  some  educational  functions.  Its  primary  purpose  is  to  build  partnerships,  and  both  the 
educational  and  military  roles  of  the  school  need  to  serve  that  objective.  This  is  not  to  say  that 
fulfilling  the  security  cooperation  role  is  all  that  matters:  Obviously,  failing  to  adequately  fulfill 
the  training  function  well  would  have  a  dramatically  negative  impact  on  U.S.  security  coopera¬ 
tion  objectives.  All  three  roles  are  important,  but  security  cooperation  should  take  precedence. 

Thus,  we  offer  the  following  six  recommendations  for  DLIELC: 


15  In  Chapter  Nine  we  fully  address  this  issue. 
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1.  Deemphasize  the  academic  character  of  the  organization.  This  currently  distracts  from 
its  main  mission  as  a  training  organization  and  an  instrument  for  security  cooperation. 

2.  Look  beyond  ELT  credentials  in  teacher  recruitment  process.  Experience  and  skills  in 
cross-cultural  communication  and  a  basic  knowledge  of  security  assistance  and  security 
cooperation  would  be  highly  valuable  qualifications  for  anyone  who  teaches  at  DLIELC. 

3.  For  all  instructors,  expand  faculty  pre-service  training  (PST)  to  provide  basic  skills  and 
understanding  in  security  cooperation.  Providing  skills  and  understanding  in  key  areas 
related  to  the  core  mission  is  an  important  step  in  building  both  mission  valence  (attrac¬ 
tiveness  of  the  mission)  and  commitment  to  the  school.16 

4.  Have  more  military  personnel  interact  with  IMSs.  Anecdotal  evidence  from  DLIELC 
sources  suggests  that  IMSs  look  for  interactions  with  U.S.  military  personnel.17  How¬ 
ever,  this  happens  rarely,  because  U.S.  military  personnel  focus  on  U.S.  students.  Thus, 
consideration  should  be  given  to  orientation  tours  of  nearby  DoD  facilities  and  units. 
“Militarizing”  to  some  extent  the  environment  of  IMSs  could  potentially  have  a  posi¬ 
tive  effect  on  students’  motivation,  facilitate  the  transition  from  DLIELC  to  FOT,  and 
reduce  disciplinary  issues.18 

5.  Consider  creating  an  alumni  program  and  modeling  this  program,  perhaps,  on  the 
George  C.  Marshall  Center  for  Security  Studies  or  other  similar  alumni  network  orga¬ 
nizations.  Such  a  program  could  include  hosting  meetings  in  the  United  States  or  in 
partner  countries,  sending  e-newsletters,  creating  alumni  chapters  in  the  respective 
partner  countries,  enabling  professional  and  social  networking  websites  to  build  on  and 
sustain  IMS  English  language  skills  and  shared  experiences  at  DLIELC.  Promotions  of 
former  DLIELC  students  could  be  highlighted,  among  other  important  news  events. 
Moreover,  it  may  be  possible  for  DLIELC  alumni  to  receive  support  from  local  organi¬ 
zations  involved  in  ELT  (e.g.,  the  State  Department,  embassies). 

6.  Reemphasize  the  importance  of  building  relationships  with  students  and  consider  new 
initiatives  such  as  developing  field  trips  (to  include  U.S.  military  units  and  facilities)  and 
cultural  activities,19  including  the  AMIGO  program,  which  needs  a  thorough  assess¬ 
ment.20  This  means  increasing  DLIELC’s  budget  accordingly  to  reflect  growing  require¬ 
ments  in  student  load;  such  readily  identifiable  and  discrete  incremental  costs  could  be 
easily  factored  into  the  cost  models  that  are  to  be  introduced  into  the  financial  manage¬ 
ment  structures  and  practices  of  DLIELC. 


16  See  Hal  G.  Rainey  and  Paula  Steinbauer,  “Galloping  Elephants:  Developing  Elements  of  a  Theory  of  Effective  Govern¬ 
ment  Organizations  P  Journal  of  Public  Administration  Research  and  Theory ,  Vol.  9,  No.  1,  1999. 

17  Another  way  to  provide  IMSs  with  a  more  military  environment  would  involve  making  IMS  life  more  similar  to  the 
U.S.  students’  life.  IMSs  could,  for  instance,  have  some  PT  or  other  types  of  drills  during  their  free  time.  Interactions  with 
DLIELC  faculty  are  an  integral  part  of  an  IMS  experience,  which,  in  turn,  impacts  the  school’s  effectiveness  in  meeting  its 
building  partnership  objectives. 

18  Currently,  disciplinary  issues  are  handled  by  the  sponsoring  military  department  rather  than  DLIELC.  Anecdotal  evi¬ 
dence  from  DLIELC  sources  suggests  that  it  takes  a  major  offense  to  get  expelled  from  DLIELC.  Breaches  of  discipline,  for 
the  most  part,  are  not  sanctioned. 

19  An  inexpensive  option  would  be  to  start  a  speaker  series  with  guest  speakers  coming  to  DLIELC  to  discuss  human  rights 
law,  civil/military  relations,  and,  more  generally,  topics  related  to  American  values. 

20  It  would  be  useful  to  find  out,  in  particular,  why  this  program  finds  an  insufficient  number  of  sponsors,  and  whether 
this  is  related  to  the  requirement  for  sponsors  to  provide  DLIELC  with  a  quarterly  Sponsorship  Report,  a  process  that  some 
prospective  participants  may  find  too  cumbersome. 


CHAPTER  SEVEN 


Assessment 


As  prescribed  in  AFJI  16-103,  DLIELC  uses  a  combination  of  internal  and  external  assessments 
to  evaluate  the  DELP.  Results  of  these  assessments  are  provided  to  a  variety  of  organizations, 
including  HQ  AETC,  which  receives  a  periodic  summary  known  as  the  balanced  scorecard. 

This  chapter  reviews  a  range  of  assessments  conducted  by  DLIELC,  including  strategic 
assessments  and  student  and  instructor  evaluations,  and  then  describes  the  results  of  our  analy¬ 
sis  of  these  assessments.  Based  on  this  analysis,  we  find  that  the  lack  of  a  formal  and  function¬ 
ing  assessment! evaluation  process  inhibits  DLIELC from  being  capable  of  defining  and  assessing  its 
own  performance.  DLIELC’s  current  processes  are  not  working,  whereas  other  training  and 
educational  units  in  AETC  are  successful.  The  chapter  concludes  with  some  specific  actions 
for  improving  the  balanced  scorecard,  as  well  as  six  recommendations  for  enabling  DLIELC  to 
establish  a  robust  overall  assessment  process.  These  recommendations  would  be  codified  in  the 
overarching  guidance  recommended  in  Chapter  Two. 


Internal  Assessments 

Student  Feedback 

To  identify  strengths  and  weaknesses  in  ELT,  DLIELC  conducts  an  EOC  survey  two  weeks 
prior  to  graduation.  This  survey  gives  students  an  opportunity  to  provide  feedback  on  several 
areas  of  ELT,  including  classroom  environment,  effectiveness  of  training,  and  use  of  technol¬ 
ogy.  Several  related  sections  address  DLIELC  operations,  tours  and  the  AMIGO  program,  and 
DLIELC  services  and  facilities  (e.g.,  the  dining  hall).  The  survey  contains  separate  sections  and 
questions  for  GET,  SET,  the  instructor  development  branch  (IDB),  and  ECHO  Company. 

Student  comment  sheets  are  available  to  students  to  provide  positive  or  negative  feedback 
to  DLIELC.  Once  the  comment  sheets  are  completed,  students  leave  them  in  one  of  the  des¬ 
ignated  drop  boxes.  The  ability  to  complete  comment  sheets  at  any  time  allows  DLIELC  to 
address  student  concerns  in  a  timely  manner. 

Instructor  Effectiveness 

Regular  evaluations  of  instructors  by  DLIELC  management  are  necessary  to  ensure  the  qual¬ 
ity  of  classroom  instruction  and  provide  useful  feedback  to  instructors.  DLIELC  1025.58  pre¬ 
scribes  the  guidelines  on  how  instructor  evaluations  are  conducted.  Instructors  are  observed 
and  formally  evaluated  twice  each  year  on  three  dimensions:  (1)  planning  and  preparation, 
(2)  techniques  of  instruction,  and  (3)  learning  environment.  As  an  administrative  tool,  the 
performance  evaluation  form  is  adequate.  However,  the  effectiveness  of  the  form  for  providing 
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developmental  feedback  to  instructors  depends  on  the  quality  of  comments  provided  by  raters 
on  the  form. 

Field  Trips 

DLIELC  also  evaluates  the  objectives  met  when  students  take  field  trips.  Field  trips  are  offered 
as  part  of  the  field  studies  program  (FSP)  to  comply  with  DoD  Instruction  5410.17.  Previously 
known  as  the  Informational  Program  (IP),  the  FSP  (see  chapter  11  of  the  JSAT)  is  voluntary, 
but  is  to  be  encouraged.  The  FSP  is  designed  to  support  four  areas  of  emphasis:  (1)  interna¬ 
tionally  recognized  human  rights  as  outlined  in  the  Universal  Declaration  of  Human  Rights;1 
(2)  the  democratic  ideals  of  an  elected  government  and  effective  civil-military  relations  that 
reinforce  that  elected  government;  (3)  the  roles  and  interrelationships  of  a  culturally,  ethni¬ 
cally,  economically,  and  socially  diverse  population  in  a  democratic  society;  and  (4)  the  U.S. 
free  enterprise  system  and  its  role  in  a  democratic  society.  To  determine  compliance  with  this 
requirement,  DFIEFC  students  complete  a  form  following  each  field  trip  indicating  if  the 
objectives  were  met. 


External  Assessments 

The  English  Fanguage  Preparedness  Survey,  formerly  the  Direct  Mail  Questionnaire,  is 
designed  to  evaluate  how  well  DFIEFC  course  objectives  and  training  serve  FOT  needs.  Spe¬ 
cifically,  the  feedback  helps  DFIEFC  determine  if  DFIEFC  graduates  have  the  necessary  lan¬ 
guage  skills  to  succeed  in  FOT.  The  FOT  instructors  evaluate  students’  language  skills  in 
several  areas — listening,  comprehension,  speaking,  reading,  and  writing.  It  should  be  noted 
that  DFIEFC  graduates  (i.e.,  students  in  FOT)  no  longer  complete  self-assessments  using  this 
survey. 

As  an  additional  external  assessment,  DFIEFC  requests  the  CEA,  an  accrediting  body 
recognized  by  the  U.S.  Department  of  Education,  to  independently  evaluate  DFIEFC.  The 
CEA  accredits  English  language  programs  using  objective  and  established  standards.  These 
standards  provide  the  foundation  for  evaluating  an  English  language  program  on  several  fac¬ 
tors,  including  its  curriculum,  faculty,  administration,  student  services,  and  student  achieve¬ 
ment.  The  CEA  first  accredited  DFIEFC  in  2000.  DFIEFC  was  reevaluated  in  2005,  receiving 
full  accreditation  for  an  additional  ten  years. 


Strategic  Assessments 

DFIEFC  periodically  provides  its  reporting  chain,  the  37th  Training  Wing,  with  a  balanced 
scorecard.  The  scorecard  presents  additional  metrics  for  evaluating  organizational  effective¬ 
ness.  These  metrics  include  DFIEFC’s  international  student  graduation  rate,  instructor  devel¬ 
opment  production,  graduates’  satisfaction  with  training  for  professional  growth,  ability  to 
respond  to  COCOM  requests  for  MTTs,  subject  matter  expert  exchanges,  FTDs,  student-to- 
instructor  ratios,  timeliness  of  hiring,  percentage  of  medical  clearances  completed,  and  sched¬ 
uling  effectiveness. 


i 


See  United  Nations  General  Assembly,  Universal  Declaration  of  Human  Rights,  December  10,  1948,  217  A  (III). 
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Evaluation  of  Student  Success 

Graduation  requirements  are  outlined  in  DLIELC  1025.25,  Chapter  1.  Specifically,  students 
must  be  scheduled  in  the  MASL,  meet  all  prerequisites,  be  in  attendance  for  at  least  18  class 
periods  (or  80  percent  of  available  class  time),  and  successfully  complete  quiz/performance 
evaluations  the  last  week  of  training,  whereupon  they  receive  a  certificate  of  graduation.  Stu¬ 
dents  must  also  achieve  graduation  requirements  specified  for  individual  MASLs  or  FOT  (e.g., 
ECL  score  required  by  FOT).  Students  failing  to  meet  any  of  the  requirements  for  graduation 
may  still  be  provided  with  a  certificate  of  attendance  or  a  certificate  of  training. 

Defining  student  success  presents  many  challenges  for  DLIELC.  A  number  of  factors 
affecting  a  student’s  ability  to  meet  graduation  requirements  are  beyond  DLIELC’s  control. 
For  example,  DLIELC  does  not  control  the  length  of  time  a  student  is  enrolled.  Students  can 
be  underprogrammed  or  have  training  time  curtailed  by  the  sponsoring  military  department. 
Consequently,  students  may  be  making  satisfactory  progress  toward  graduation  but  are  not 
provided  with  sufficient  time  to  meet  the  English  requirements  for  FOT.  Another  concern 
found  in  defining  success  occurs  when  a  student’s  graduation  requirements  change  during  the 
course  of  training.  This  may  occur,  for  example,  when  a  student’s  FOT  changes  and  the  cor¬ 
responding  English  requirements  are  different. 

Improving  Assessment  Metrics 

The  most  recent  balanced  scorecard  (as  of  December  31,  2010)  indicated  a  few  areas  of  con¬ 
cern,  including  satisfaction  of  instructor  development  training  among  graduates,2  student-to- 
faculty  ratio,  hiring  timeline  for  instructors,  and  completion  of  student  medical  clearances. 
The  percentage  of  DLIELC  instructor  development  graduates  who  were  satisfied  with  train¬ 
ing  for  professional  growth  (74  percent)  was  slightly  below  the  goal  of  85  percent.  Additional 
analysis  by  DLIELC  of  this  issue  suggested  that  the  lower  score  could  be  partly  attributed  to  a 
particular  course  that  used  a  new  instructor. 

Concerns  directly  related  to  the  content  and  breadth  of  the  assessments  were  also  iden¬ 
tified.  For  example,  although  the  assessment  of  the  FSP  shows  that  it  is  meeting  minimum 
requirements,  the  assessment  is  extremely  terse  and  does  not  provide  enough  information  to 
determine  the  quality  or  quantity  of  information  learned  during  held  trips. 

Other  assessments,  such  as  the  EOC  survey,  reflect  DLIELC’s  complex  identity  as  an 
academic,  military,  and  security  cooperation  institution.  Questions  in  the  EOC  survey  sug¬ 
gest  that  DLIELC’s  academic  function  is  the  primary  focus  of  this  assessment.  The  BP  role  (a 
subset  of  security  cooperation)  of  the  school  is  somehow  reflected  in  the  numerous  questions 
that  assess  the  quality  of  the  support  staff  and  facilities  at  DLIELC.  It  is  also  covered  by  more- 
general  questions  about  how  pleasant,  welcoming,  and  respectful  IMSs  find  their  environment 
at  DLIELC  to  be.  These  questions  are  useful  for  DLIELC  to  find  out  whether  any  element 
(be  it  teaching  staff,  support  staff,  infrastructures,  programs)  is  inadequate  and  needs  to  be 
improved.  In  terms  of  evaluating  the  BP  impact  of  DLIELC,  however,  such  questions  provide 
only  a  short-term  perspective.  Although  they  describe  how  satisfied  students  are  with  their  time 
at  DLIELC,  the  question  is  whether  their  experiences  will  durably  affect  their  perception  of 
the  United  States  and  lead  them,  over  the  course  of  their  career,  to  promote  links  (at  whatever 


Students  attending  instructor  development  are  those  who  will  be  or  are  English  language  instructors  in  their  country. 
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level  that  may  be)  between  their  country  and  the  United  States.  BP,  like  security  cooperation, 
is  a  complex  notion.  A  previous  RAND  study  noted  the  following: 

Security  cooperation  programs  are  designed  to  promote  shared  values  and  threat  percep¬ 
tions,  so  that  others  will  be  more  likely  to  identify  security  interests  in  common  with  those 
of  the  United  States;  shared  technology  and  procedures,  so  that  others  will  be  more  able 
to  protect  common  interests;  and  the  freedom  to  devote  Air  Force  resources  to  the  highest 
priority  missions.  Although  it  is  important  for  partners  to  become  technically  more  capable 
as  a  result  of  this  cooperation,  the  U.S.  Air  Force  emphasizes  building  enduring  relation¬ 
ships  through  shared  skills  and  the  trust  that  emerges  from  gaining  confidence  in  these 
shared  skills.3 

This  long-term  dimension,  and  how  DLIELC  specifically  contributes  to  it,  is  largely 
overlooked  by  DLIELC’s  current  assessment  tools  (for  possible  BP  metrics;  see  Appendix  A). 
Additional  assessment  approaches  are  needed  to  follow  up  with  students  after  FOT  and  upon 
returning  to  their  homelands. 

As  a  strategic  assessment,  the  balanced  scorecard  provides  useful  measures  in  several  rel¬ 
evant  areas.  Elowever,  not  all  perspectives  are  clearly  represented.  As  part  of  a  well-designed 
performance  measurement  framework,  Kaplan  and  Norton  (1996)  suggest  that  organiza¬ 
tions  should  develop  performance  measures  for  each  of  four  areas:  (1)  financial,  (2)  customer, 
(3)  internal  business  processes,  and  (4)  learning  and  growth.4  DLIELC  measures  largely 
emphasize  the  customer  and  the  internal  business  process  aspects.  The  focus  on  these  two 
areas  is  consistent  with  the  primary  mission  of  DLIELC  to  train  international  students  in 
English.  The  remaining  two  perspectives,  however,  are  not  well  represented  in  DLIELC’s  bal¬ 
anced  scorecard.  The  learning  and  growth  perspective  could  be  captured  in  part  by  continuous 
improvement  recommendations;  however,  benchmarks  for  this  process  were  not  included  as 
part  of  the  balanced  scorecard.  The  financial  perspective  was  also  not  represented  on  the  bal¬ 
anced  scorecard  provided  to  the  37th  Training  Wing.  Overall,  the  following  factors  and  related 
metrics  should  be  included  in  a  revised  balanced  scorecard  methodology: 

1.  Financial.  Core  dimensions  include  operational  expenses,  revenue  (e.g.,  American  Lan¬ 
guage  Course),  and  requirements  (e.g.,  POM). 

2.  Internal  business  processes.  Core  dimensions  include  student  management  (e.g.,  pro¬ 
gramming,  discipline),  curriculum  development,  instruction  and  testing  (e.g.,  test  vali¬ 
dation),  physical  infrastructure  (e.g.,  information  technology  support,  classroom  qual¬ 
ity  and  availability,  housing),  mission  support  (e.g.,  timeliness  of  hiring),  and  resource 
utilization  (e.g.,  time  between  qualifying  for  FOT  and  attendance  at  FOT). 

3.  Customers.  Core  dimensions  include  partner  capacity  (e.g.,  operable  systems,  student 
commitment  and  satisfaction),  English  proficiency  (e.g.,  ECL  scores,  confidence  in 
communicating),  cultural  awareness,  and  student  success  (e.g.,  training  readiness  for 
FOT,  alumni  survey). 


3  Jennifer  D.  P.  Moroney,  Kim  Cragin,  Eric  Stephen  Gons,  Beth  Grill,  John  E.  Peters,  and  Rachel  M.  Swanger,  Interna¬ 
tional  Cooperation  with  Partner  Air  Forces,  Santa  Monica,  Calif.:  RAND  Corporation,  MG-790-AF,  2009,  p.  5. 

4  Robert  S.  Kaplan  and  David  P.  Norton,  Translating  Strategy  into  Action:  The  Balanced  Scorecard,  Boston,  Mass.:  Harvard 
Business  School  Press,  1996. 
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4.  Learning  and  growth.  Core  dimensions  include  FOT  site  feedback,  internal  feedback 
from  stalf  and  instructors,  instructor  quality  and  development  (e.g.,  performance  rat¬ 
ings,  feedback  and  development  plans),  and  staff  development  (e.g.,  support  staff  com¬ 
mitment  and  satisfaction). 

A  separate  concern  relates  to  the  FSP.  Although  the  international  training  summary  slides 
for  the  balanced  scorecard  referenced  an  American  culture  survey,  DLIELC  indicated  that  this 
particular  survey  was  abandoned  many  years  ago.  According  to  responses  from  DLIELC,  stu¬ 
dents  had  a  difficult  time  accurately  completing  the  survey,  resulting  in  poor  quality  responses. 
Currently,  DLIELC  uses  students’  dichotomous  responses  on  Form  5140.2(D)  following  a 
held  trip  to  determine  if  the  selected  objective  was  met.  Specifically,  students  circle  one  of  the 
listed  objectives  for  the  held  trip  and  then  indicate  if  that  specihc  objective  was  met.  Although 
Form  5140.2(D)  is  short  and  simple  and  meets  the  requirement  of  the  DoD  FSP,  the  subjec¬ 
tive  responses  provided  by  students  may  not  accurately  reflect  students’  learning  of  U.S.  cul¬ 
ture.  This  is  also  true  of  the  two  EOC  survey  questions  relating  to  held  trips.  Students  are 
asked  to  evaluate  the  following  statement:  “I  learned  about  [hjuman  [fights  and  the  American 
[djemocratic  way  of  life  through  weekday/weekend  held  trips.”  These  statements  seem  too  gen¬ 
eral  for  students  to  provide  a  clear  answer:  About  a  hfth  of  them  neither  agreed  nor  disagreed 
by  providing  a  neutral  response.  Overall,  current  assessments  and  poorly  designed  surveys  do 
not  make  it  possible  to  identify  how  much  students  have  really  learned  or  which  events  facili¬ 
tate  understanding  of  U.S.  culture. 


Recommendations 

DLIELC  lacks  a  fully  functioning  assessment  process.  As  noted,  the  lack  of  a  formal  and  func¬ 
tioning  assessment/evaluation  process  inhibits  DLIELC  from  being  capable  of  dehning  targets 
and  assessing  its  own  performance.  This  chapter,  in  examining  both  student  and  instructor 
evaluations,  as  well  as  assessments  reporting  to  other  entities,  considers  three  key  elements  of 
a  comprehensive  system  of  assessments.  The  existing  guidance  does  not  mandate  any  specihc 
approach  to  assessment,  nor  does  it  hold  DLIELC  accountable  to  FIQAETC.  Nonetheless,  the 
processes  and  resources  available  within  its  current  structures  could  form  the  foundation  of  an 
effective  assessment  system  if  properly  leveraged. 

The  following  recommendations  for  DLIELC  aim  at  addressing  this  issue  and  providing 
DLIELC  with  a  more  systematic  way  to  assess  its  own  performance.  The  hrst  three  recommen¬ 
dations  relate  to  student  and  instructor  evaluations: 

•  Leverage  AETC’s  existing  training  assessment  process,  including  conformance  to  train¬ 
ing  development  and  evaluation  policies,  such  as  those  contained  in  AETC  Instruction 
36-2201,  Technical  and  Basic  Military  Training  Evaluation. 

•  Explore  standard  processes  for  training  development  and  assessment,  thereby  gaining 
access  to  the  lessons  observed  by  similar  training  organizations.  Conforming  to  existing 
practices  will  put  DLIELC  in  a  position  to  leverage  the  experience  and  resources  of  the 
full  AETC  staff,  including  the  AETC  Inspector  General. 

•  Improve  assessment  of  the  FSP  and  coordinate  with  the  Air  Force  Culture  and  Language 
Center  to  develop  measures  for  assessing  knowledge  and  comprehension  of  U.S.  culture. 
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The  next  three  recommendations  relate  to  the  balanced  scorecard: 

•  Revise  the  balanced  scorecard  to  fully  represent  each  of  the  four  performance  measure 
perspectives  discussed  above. 

•  Reevaluate  current  metrics  to  ensure  that  they  are  clearly  written;  relevant;  inclusive  of 
academic,  military,  and  BP  objectives;  and  primarily  under  DLIELC’s  control. 

•  Include  specific  metrics  geared  toward  measuring  the  long-term  BP  impact.  Assessing 
long-term  BP  impact  is  difficult,  but  a  solution  could  include  the  following  two  steps: 

-  Set  up  a  pre-  and  post-training  survey  and/or  quiz  focusing  on  U.S.  culture  and  values. 
Much  good  work  done  has  been  done  in  this  field  by,  for  example,  the  World  Values 
Survey,  Pew  Trust,  the  International  Institute  of  Education,  and  Gallup,  all  of  which 
should  be  consulted  when  designing  these  surveys.  The  degree  to  which  students 
improve  in  these  domains  between  the  beginning  and  the  end  of  their  training  could 
be  matched  with  their  country  of  origin,  length  of  stay  at  DLIELC,  initial  ECL  score, 
course  attended,  and  the  specific  field  visits  made.  Progress  could  be  monitored  over 
the  years. 

-  Hold  focus  groups  with  students,  using  well-defined  focus  group  protocol,  to  find  out 
how  their  knowledge  and  perceptions  of  the  United  States  have  changed  during  their 
time  at  DLIELC.  Over  time,  insights  from  these  focus  groups  could  also  be  used  to 
refine  EOC  and  field  visit  surveys. 


CHAPTER  EIGHT 

Manpower 


Having  adequate  manpower  and  the  ability  to  address  variable  demand,  particularly  in  a 
dynamic  and  difficult-to-predict  training  environment,  is  essential.  Possessing  the  relevant 
mechanisms  to  quickly  and  reliably  gain  access  to  the  right  employees  is  critical  to  maintain¬ 
ing  effective  operations,  regardless  of  the  situation.  In  the  case  of  a  training  organization  like 
DLIELC,  where  resource  demands  increase  in  direct  proportion  to  the  student  load,  man¬ 
power  becomes  one  of  the  most  important  variables  in  ensuring  that  the  organization  contin¬ 
ues  to  function  effectively. 

This  chapter  examines  the  current  hiring  process  and  looks  at  utilization  of  instructors, 
instructor  qualifications,  the  military  environment,  and  meeting  short-term  demand.  Based  on 
our  analysis,  we  find  that  the  hiring  process  is  unable  to  respond  quickly  to  short-term  and  cyclical 
demand.  The  chapter  concludes  with  recommendations  for  addressing  this  challenge. 


Current  Hiring  Process 

The  process  by  which  DLIELC  hires  new  instructional  staff  is  laboriously  long  and  is  a  con¬ 
tributing  factor  to  the  institution  not  being  able  to  respond  in  a  timely  manner  to  variations  in 
demand  for  ELT.  Because  of  the  numerous  levels  of  approvals  needed  for  a  new  hire,  recruiting 
a  new  instructor  reportedly  averages  over  six  months. 


Utilization  of  Instructors 

The  current  teaching  faculty  of  DLIELC  consists  of  172  faculty  instructors.  Of  these,  59  are 
not  engaged  in  full-time  instruction  and  are,  instead,  detailed  to  curriculum  development, 
MTTs,  and  other  duties.  This  creates,  at  times,  a  shortage  of  teaching  staff.  We  were  unable 
to  ascertain  how  DLIELC  can  justify  the  dedication  of  such  a  large  number  of  instructors  to 
curriculum  development,  especially  given  that  the  school  is  replacing  modules  on  average  every 
three  to  five  years,  despite  this  large  investment  of  resources.  Given  the  highly  competitive 
nature  of  the  foreign  language  training  market,  more  cost-effective  means  should  be  explored 
by  HQAETC  leadership  and  DLIELC  management.1  In  cases  where  curriculum  development 
can  be  effectively  outsourced  (e.g.,  GET),  DLIELC  can  expand  the  use  of  dedicated  curricu- 


1  This  should  consist  of  purchasing  existing  commercial  content  packages  and  instructional  tools  such  as  software  where 
they  meet  DLIELC’s  ELT  requirements. 
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lum  developers  as  a  reserve  pool  of  language  instructors  to  be  utilized  in  response  to  short-term 
higher  demand  (although  variable  demand  for  ELT  could  be  more  efficiently  met  through  con¬ 
tract  instructors;  see  below).  Where  there  are  requirements  to  develop  stand-alone  specialized 
curricula,  these  should  be  coordinated  by  the  actual  experts  in  that  particular  technological  or 
training  field,  with  guidance  from  English  language  instructors. 


Qualifications  of  Instructors 

DLIELC  civil  service  instructors  must  meet  certain  educational  requirements  to  be  consid¬ 
ered  for  employment.2  DLIELC  uses  a  category  ratings  system,  which  indicates  that  the  best- 
qualified  instructors  have  a  combination  of  education  in  ESL/EFL  and  professional  experience 
teaching  ESL/EFL  to  adults.  While  ostensibly  a  reasonable  requirement,  we  find  that  the  rigid 
application  of  these  standards  might  be  counterproductive.  One  of  DLIELC’s  critical  tasks  is 
to  educate  selected  students  in  an  English  technical  vocabulary  to  prepare  them  for  FOT.  Such 
knowledge  is  an  invaluable  tool  to  enable  effective  and  efficient  FOT.  However,  the  school 
assumes  that  an  academically  credentialed  instructor  of  English  language  who  lacks  technical 
background  or  experience  in  a  specialized  field  can  be  as  effective  in  providing  instruction  in 
technical  language  as  a  retired  NCO  who  does  not  possess  a  college  degree  in  ESL/EFL.  We 
suggest  that  a  blended  team  approach,  which  combines  people  with  technical  backgrounds  and 
individuals  with  ESL/EFL  education,  may  yield  stronger  outcomes  for  both  building  security 
cooperation  and  developing  English  language  skills.  While  it  is  difficult  for  DLIELC  to  initiate 
changes  in  this  field  absent  revisions  to  DoD  and  U.S.  Office  of  Personnel  Management  regu¬ 
lations,  other  agencies  are  exploring  creative  options.  For  example,  the  Department  of  Veteran 
Affairs  is  creating  a  hybrid  system  using  U.S.  Code,  Title  38. 


Military  Environment 

We  are  sensitive  to  the  reality  that  it  is  implicit  in  DLIELC’s  mission  to  create  relationships 
with  students  as  a  way  to  introduce  them  to  life  in  the  United  States.  Moreover,  DLIELC  has 
an  opportunity  to  build  a  relationship  between  students  and  the  U.S.  armed  forces.  Because 
DLIELC  is  a  military  establishment,  one  might  expect  to  see  a  larger  presence  of  U.S.  armed 
forces  personnel  as  instructional  staff.  Despite  the  potential  benefits,  however,  the  military 
departments  may  be  reluctant  to  provide  personnel  to  DLIELC  to  serve  as  language  instructors. 

Two  options  constitute  possible  solutions.  First,  students  at  DLIELC  are  financed  through 
security  assistance  or  security  cooperation  funding;  under  the  provisions  of  U.S.  legislation  and 
the  Security  Assistance  Management  Manual  (DoD,  2003),  HQAETC  leadership  and  DLIELC 
management  should  ensure  that  the  military  departments  are  fully  reimbursed  for  the  acceler¬ 
ated  cost  of  military  personnel  assigned  to  DLIELC.  Second,  although  likely  suboptimal,  HQ 
AETC  leadership  and  DLIELC  management  could  explore  innovative  options  (either  through 
the  authorities  provided  in  U.S.  Code,  Title  10,  Section  1595,  and/or  contracting)  of  hiring 
retired  military  personnel,  particularly  those  who  possess  an  intimate  knowledge  of  contempo¬ 
rary  technical  language.  Just  as  DLIFLC  provides  its  foreign  instructors  (who  do  not  necessar- 
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ily  hold  a  teaching  degree  in  their  native  language)* 1 2 3  with  extensive  training  prior  to  taking  on  a 
class,  DLIELC  could  develop  an  annual  training  program  to  retrain  retired  military  personnel 
to  become  instructors. 


Meeting  Short-Term  Demand 

Innovative  Hiring  Practices 

HQ  AETC  leadership  and  DLIELC  management  may  wish  to  take  advantage  of  the  provi¬ 
sions  of  Title  10,  Section  1595,  regarding  the  Secretary  of  Defense’s  authority  to  hire  civil¬ 
ians  at  DLIELC  on  a  term  basis.  The  language  in  law  covers  the  ability  of  the  Secretary  of 
Defense  to  make  term  appointments  at  the  “English  Language  Center  of  the  Defense  Lan¬ 
guage  Institute.”4 

If  such  a  course  of  action  were  followed,  DLIEC  could  make  local  hiring  decisions  on  an 
“excepted  service”  basis,  thereby  precluding  the  necessity  to  develop  and  adhere  to  a  certifica¬ 
tion  list  of  candidates,  as  is  the  case  with  standard  hiring  practices.  Although  a  shift  in  hiring 
authority  would  have  to  be  executed  in  full  accordance  with  Air  Lorce  regulations  and  AETC’s 
standing  operating  procedures  for  the  management  of  civilian  employment,  other  DoD  educa¬ 
tional  institutions  (e.g.,  Air  University)  have  demonstrated  that  this  can  be  done  successfully. 

Alternatively,  DLIELC  could  explore  the  use  of  consultants  and  experts,  as  authorized 
by  Air  Lorce  Manual  36-203,  Staffing  Civilian  Positions,  2002.  Such  an  approach  could  pro¬ 
vide  a  ready  pool  of  language  professionals  “on  call”  and  ready  to  come  to  the  school’s  aid 
when  needed.  Importantly,  the  lack  of  overhead  costs  and  long-term  commitment  makes  this 
an  attractive  solution  from  a  financial  standpoint.  Similarly,  DLIELC  could  also  explore  the 
hiring  of  instructors  on  a  less-than-full-time  basis.  This  type  of  arrangement  could  also  lead  to 
a  low-cost  pool  of  professionals  who  could  augment  the  full-time  faculty  simply  by  increasing 
their  hours  when  required.5 


3  According  to  DLIFLC’s  website,  “In  addition  to  the  Academic  Rank  qualifications,  applicants  must  have  near  native 
language  proficiency  in  all  skills,  in  addition  to  strong  English  skills.  Language  testing  will  be  required  for  recommended 
candidates.  A  four-year  accredited  university  degree  is  the  minimum  requirement.  Education  related  to  foreign  language 
education,  linguistics  or  MATFL/TESOL,  etc.,  is  preferred ”  [emphasis  added]. 

4  U.S.  Code,  Title  10,  Subtitle  A,  Part  II,  Chapter  81,  Section  1595,  “Civilian  faculty  members  at  certain  Department  of 
Defense  schools:  employment  and  compensation.” 

(a)  Authority  of  Secretary. — The  Secretary  of  Defense  may  employ  as  many  civilians  as  professors,  instructors,  and  lecturers 
at  the  institutions  specified  in  subsection  (c)  as  the  Secretary  considers  necessary. 

(b)  Compensation  of  Faculty  Members.  —  The  compensation  of  persons  employed  under  this  section  shall  be  as  prescribed 
by  the  Secretary. 

(c)  Covered  Institutions. — This  section  applies  with  respect  to  the  following  institutions  of  the  Department  of  Defense: 

(1)  The  National  Defense  University. 

(2)  The  Foreign  Language  Center  of  the  Defense  Language  Institute. 

(3)  The  English  Language  Center  of  the  Defense  Language  Institute. 

(4)  The  Western  Hemisphere  Institute  for  Security  Cooperation. 

(d)  Application  to  Faculty  Members  at  NDU. — In  the  case  of  the  National  Defense  University,  this  section  applies  with 
respect  to  persons  selected  by  the  Secretary  for  employment  as  professors,  instructors,  and  lecturers  at  the  National  Defense 
University  after  February  27,  1990. 

5  See  DoD  Directive  1400. 25-M,  Civilian  Personnel  Management,  2009,  Subchapter  340. 


58  Optimizing  the  Defense  Language  Institute  English  Language  Center 


Finally,  as  mentioned  in  other  chapters,  military  reservists  could  be  a  short-term  fit  that 
would  also  provide  a  military  presence  and  an  intimate  knowledge  of  contemporary  technical 
language. 

Contracting  Instruction 

Given  the  more  recent  increase  in  demand  for  English  language  training,  DLIELC  has  been 
able  to  use  an  existing  General  Services  Administration  contract  vehicle  to  hire  contract  Eng¬ 
lish  language  instructors  to  meet  variable  demand.6  However,  a  dedicated  contract  vehicle 
optimally  developed  to  meet  DLIELC’s  unique  character  and  labor  requirements  would  be 
preferable. 


Case  Study  Observations 

The  Canadian  Forces  English/French  language  instructional  program  uses  government 
employees  as  managers  for  the  program  but  employs  contracted  instruction  to  meet  its  variable 
demand  requirements.  Within  the  security  cooperation  community,  such  arrangements  are 
increasingly  the  norm  (CCMR,  DLIFLC,  PLTCE,  and  COE-DMHA),  because  they  enable 
these  organizations  to  meet  variable  demand  in  a  very  cost-efficient  and  content-effective 
manner. 


Recommendations 

As  noted  above,  we  found  that  DLIELC’s  current  hiring  process  is  inefficient,  inherently  inflex¬ 
ible,  and  unable  to  respond  to  short-term  and  cyclical  demand.  The  following  recommendations 
address  these  issues  by  addressing  policy  alternatives  and  innovative  practices.  In  terms  of 
policy  alternatives: 

•  Utilize  civilian  academic  hire  provisions  in  U.S.  Code,  Title  10. 

•  End  the  current  practice  of  curriculum  development.  Explore  commercially  available,  off- 
the-shelf  curriculum/instructional  techniques. 

In  terms  of  innovative  practices: 

•  Establish  an  IDIQ  contract  vehicle  to  hire  ELT  instructors,  as  required.  Have  specialized 
military/technical  language  curriculum  requirements  executed  and  further  supported  by 
knowledgeable  experts  in  those  fields  and  recover  the  cost  of  their  labor  from  the  request¬ 
ing  client. 

•  Emphasize  the  importance  of  building  partnerships  to  obtain  more  military  department 
personnel.  Such  assignments  must  be  on  a  full  cost-recovery  basis  with  the  services.  Fur- 


This  should  be  considered  a  fortuitous  short-term  solution.  DLIELC  should  have  its  own  IDIQ  contract  for  temporary 


instructors. 
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thermore,  HQ  AETC  leadership  and  DLIELC  management  should  explore  hiring  or 
contracting  for  retired  military  personnel. 

•  Use  experts  and  consultants  as  a  ready  pool  of  language  professionals  who  could  be 
brought  on  to  quickly  fill  gaps  because  of  unforeseen  student  load  fluctuations. 


CHAPTER  NINE 


Organization  and  Advocacy 


An  organizational  structure  with  clear  lines  of  authority  ensures  that  all  personnel  within 
the  organization  have  clear  expectations  of  their  respective  roles.  In  an  effective  organization, 
employees  know  where  to  go  to  get  their  job  done  and  understand  their  chain  of  command. 
Office  directors  have  a  clear  sense  of  purpose  and  can  easily  identify  the  roles  of  others  in  the 
organization  and  can  ensure  that  the  path  to  the  highest  levels  of  the  chain  of  command  is 
clear,  widely  understood,  and  accessible. 

Strong  institutional  advocacy  ensures  that  resourcing  issues  are  addressed  at  the  right 
level  and  that  the  institution  has  a  voice  and  a  champion  when  questions  arise.  Institutional 
advocacy  is  not  easy.  In  the  case  of  an  interservice  organization  like  DLIELC,  there  are  many 
potential  masters;  unless  a  single  advocate  is  clearly  identified,  failures  are  likely  to  occur. 

Based  on  our  analysis,  we  find  that  the  organization  struggles  to  meet  variable  demand. 
DLIELC  officials  told  us  that  reporting  requirements  can  be  cumbersome  and  confusing  to 
incoming  students.  Importantly,  the  organization  has  lacked  an  institutional  advocate,  result¬ 
ing  in  a  lack  of  focused  oversight  within  DoD  as  it  relates  to  ensuring  the  effective  and  efficient 
execution  of  ELT. 

In  this  chapter,  we  detail  where  these  findings  come  from  and  provide  some  recommen¬ 
dations  to  address  them. 


Structure  and  Advocacy 

After  careful  investigation  and  review  of  DLIELC’s  management  structures  and  linkages  to 
national-level  policy,  it  is  clear  that  the  organization  has  been  ill  served  by  the  management 
oversight  structure.  As  result,  DLIELC  suffers  from  three  major  organizational  weaknesses: 

•  isolation  from  HQAETC’s  institutional  managerial  oversight 

•  lack  of  advocacy  to  support  long-term  development 

•  isolation  from  national-level  security  cooperation  policy. 

Recently,  HQ  Air  Force  has  become  actively  engaged  with  greater  oversight  and  a  view 
toward  optimizing  DLIELC’s  performance.  Obviously,  the  sui  generis  nature  of  the  organi¬ 
zation  has  worked  against  its  full  integration  within  the  HQAETC  community.  We  believe, 
however,  that  DLIELC  would  function  better  if  it  were  more  fully  integrated  into  that  educa¬ 
tion  and  training  community  and  subject  to  its  institutional  oversight  and  performance  evalu¬ 
ation  methods. 
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Since  DLIELC  provides  students  to  the  technical  training  schoolhouses  of  the  Air  Force, 
Army,  and  Navy,  its  leadership  must  be  graded  on  its  ability  to  provide  students  with  sufficient 
English  fluency  to  do  the  FOT  as  scheduled.  Failure  to  provide  qualified  students  on  time 
causes  training  seats  to  go  empty,  resulting  in  a  potential  loss  of  production  to  FOT  provid¬ 
ers.1  While  AETC  is  not  measured  by  the  production  of  foreign  students,  setting  aside  FOT 
seats  limits  the  opportunity  to  use  those  same  seats  for  U.S.  students.  Each  service  has  the 
same  requirement,  and  2AF  would  have  the  ultimate  responsibility  for  meeting  other  service 
requirements.  Consequently,  this  is  a  strong  argument  that  DFIEFC  should  report  to  a  mili¬ 
tary  department. 

As  for  the  lack  of  advocacy,  recent  POM  deliberations  have  shown  that  DFIEFC  is  dis¬ 
connected  from  an  established  structure  that  can  provide  advocacy  and  support.  Moreover,  ties 
to  a  clear  management  structure  are  needed  to  ensure  that  DFIEFC  is  accountable,  through 
the  chain  of  command,  for  its  security  cooperation  mission.  The  absence  of  a  proper  require¬ 
ments  identification  process  highlights  one  of  the  consequences  of  the  disconnected  manage¬ 
ment  process.  Recently,  HQ  AF/A1DG  hired  personnel  to  specifically  create  and  manage  a 
requirements  process  and  to  pursue  advocacy  for  DFIEFC  at  the  Air  Staff  level.  These  are  wel¬ 
come  developments  and  bode  well  for  correcting  advocacy  concerns  in  the  future. 

Of  concern  to  us  is  that  while  there  is  great  merit  in  aligning  DFIEFC  more  closely  to 
HQ  AETC  and  AF/A1DG,  this  by  itself  will  not  provide  DFIEFC  a  closer  relationship  to  the 
security  cooperation  community  so  as  to  have  improved  visibility  of  impending  demand  and 
gain  an  institutional  voice  in  affecting  the  inflow  (in  size  and  time)  of  students  the  better  to 
execute  its  mission.  Currently,  DFIEFC’s  EA  is  the  AF/A1DG.  One  issue  with  this  approach 
is  that  AF/A1DG  and  parallel  staffs  elsewhere  are  not  established  in  the  security  cooperation 
community,  either  within  the  USAF  or  more  broadly,  with  the  COCOMs,  Joint  Staff,  and 
OSD/Policy.  AF/A1DG  officials  have  generally  not  attended  conferences,  planning  meetings, 
and  workshops  focused  on  security  cooperation  issues.  Such  events  are  a  primary  venue  for 
organizational  advocacy  to  take  place.  HQ  AF/A1DG  has  suggested  a  memorandum  of  agree¬ 
ment  (MOA)  between  the  principal  interested  parties  as  a  means  of  ensuring  the  right  organi¬ 
zations  have  a  role  in  managing  DFIEFC.  We  believe  this  approach  holds  promise,  especially 
since  security  cooperation  considerations  are  very  important  when  addressing  discipline,  aca¬ 
demic  progress,  and  general  student  care. 

In  the  case  of  discipline  issues,  the  foreign  culture  and  sensitivity  of  bilateral  security 
relationships  between  the  United  States  and  each  particular  nation  requires  a  well-informed 
“diplomatic  touch”  and  special  care  in  handling  these  delicate  issues  without  causing  undue 
embarrassment  or  harm  in  foreign  relations.  Additionally,  IMSs  need  more  care  than  typical 
U.S.  students  arriving  at  a  training  base.  The  foreign  students  are  entering  a  new  culture  that 
in  many  cases  is  completely  different  from  their  home  culture.  Every  student  should  be  greeted 
at  the  airport,  and  preparations  should  be  made  to  provide  for  the  first  few  weeks  of  a  student’s 
stay  at  Fackland  AFB. 

Overall,  we  found  that  of  the  organizations  examined  in  our  case  study  analysis,  those 
reporting  either  directly  or  indirectly  to  DSCA  enjoy  a  clear  advocate  in  DSCA  and  generally 
have  a  better  understanding  of  variations  in  demand  for  professional  services  because  of  their 
integration  into  the  security  cooperation  community.  Indeed,  residing  in  a  military  depart- 


1  The  same  would  be  true  for  the  other  military  departments. 


Organization  and  Advocacy  63 


ment,  but  adhering  to  DSCA/security  cooperation  guidance  is  not  mutually  exclusive,  as  many 
BP  organizations  have  proven  this  to  be  possible.  Moreover,  this  model  is  reinforced  when  we 
recall  that  DLIELC  is  financed  on  a  reimbursable  basis,  versus  being  fully  funded  through  the 
POM  process.  These  organizations  also  have  close  financial  reporting  ties  to  DSCA  and  its 
oversight  structure.  The  fact  that  these  schools  have  a  close  connection  to  policy  has  led  them 
to  develop  effective  procedures  that  capture  all  fixed  and  variable  costs  associated  with  the 
organization’s  operation  and  recover  them  from  the  client.  Not  only  would  DLIELC  benefit 
from  a  closer  tie  to  the  security  cooperation  community  to  help  it  solve  the  issue  of  predicting 
variable  demand,  but  such  a  closer  relationship  with  DSCA  might  well  also  bring  improved 
cost-recovery  practices  to  DLIELC.  AL/AlDG’s  MOA  should  address  these  concerns  as  well. 

Ultimately,  there  are  two  conflicting  demands — the  need  to  provide  IMSs  in  a  timely 
manner  for  POT  and  the  need  to  effectively  care  and  provide  for  students  to  meet  secu¬ 
rity  cooperation  goals.  The  lack  of  adherence  to  a  military  department’s  management  rules 
and  regulations  furthers  neither  requirement.  While  it  might  work  to  place  DLIELC  under 
AETC/IA  (reporting  to  AETC  and  a  part  of  the  security  cooperation  community),  this  solu¬ 
tion  would  produce  a  headquarters  staff  organization  that  would  not  be  optimally  set  up  to 
oversee  an  executing  organization.  Prom  RAND’s  perspective,  keeping  DLIELC  under  AETC 
and  adhering  to  AETC’s  training  methods  and  standards  while  taking  security  cooperation 
guidance  from  DSCA  (through  an  MoA),  should  provide  an  acceptable  solution  to  all  key 
stakeholders. 


Meeting  Variable  Demand 

DLIELC’s  demand  signals  point  to  a  demand  function  that  is  anything  but  predictable  over 
the  medium  to  longer  term.  To  all  appearances,  DLIELC  is  organizationally  rigid  and  not  suf¬ 
ficiently  flexible  to  meet  variable  demand  signals.  The  reality  of  living  in  a  variable-demand 
environment  should  be  acknowledged.  Basic  assumptions  about  the  organization’s  structures 
and  procedures  need  to  be  reviewed  and  readjusted  as  required  to  meet  this  condition.  In 
Chapter  Three,  we  discussed  an  approach  that  provides  a  flexible  solution  to  meeting  variable 
demand,  and  in  Chapter  Eight  we  addressed  ways  to  deal  with  this  through  manpower  adjust¬ 
ments.  It  is  not  clear  that  a  different  organizational  structure  would  improve  performance,  but 
it  is  likely  that  without  higher-level  involvement,  the  challenges  confronting  DLIELC  will  be 
difficult  to  overcome.  DLIELC  operates  freely  and  with  little  oversight.  Whatever  organiza¬ 
tional  structure  emerges,  timetables  and  measures  of  performance  are  required  to  ensure  action 
and  compliance. 


Obtaining  Advocacy  in  the  Security  Cooperation  Community 

As  indicated  above,  organizations  reporting  either  directly  or  indirectly  to  DSCA  have  a  clear 
advocate  and  generally  a  better  understanding  of  demand  because  of  their  integration  into  the 
security  cooperation  community.  Similarly,  the  UK  Defence  School  of  Languages/English 
Language  Wing  is  established  on  the  clear  principle  of  supporting  the  Ministry  of  Defence’s 
policy  priorities  as  cost-efficiently  as  possible.  In  the  case  of  the  Canadian  Lorce’s  English/ 
Erench  language  training  program,  the  management  and  “ownership”  of  the  language  instruc- 
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tion  content  remains  under  the  control  of  the  Canadian  Forces;  the  actual  foreign  language 
training  is  largely  executed  through  contract  faculty. 


Recommendations 

As  noted  above,  based  on  our  findings,  DLIELC  struggles  to  meet  variable  demand  and  find 
the  right  balance  of  identities.  Until  recently,  it  lacked  a  strong  policy  champion.  This  situation 
has  produced  internal  organizational  pathologies  that  are  impeding  DLIELC  from  reaching 
its  full  potential  in  terms  of  effective  training  and  on  a  fully  cost-reimbursable  basis.  HQAF/ 
A1DG  is  already  taking  steps  to  address  these  issues,  but  the  following  recommendations  can 
help  ensure  their  success. 

In  terms  of  purely  organizational  issues,  DLIELC  should  adopt  greater  managerial  flex¬ 
ibility  to  meet  variable  demand  through  the  adoption  of  new  labor  delivery  options  (as  dis¬ 
cussed  in  the  previous  chapter  on  manpower).  In  terms  of  advocacy,  the  situation  is  more  com¬ 
plex,  but,  at  a  minimum,  DLIELC  should 

•  ensure  that  OSD  and  USAF  leadership  agree  on  clarified  oversight  responsibilities  and 
functions  of  DLIELC 

•  ensure  that  the  advocate  or  champion  within  DoD  is  actively  involved  in  the  require¬ 
ments  process.  AF/A1DG  has  a  plan  to  do  this;  from  an  execution  perspective,  the  plan 
would  be  most  easily  implemented. 

Finally,  should  OSD  and  USAF  leadership  agree  that  DLIELC  remain  under  the  man¬ 
agement  of  the  USAF,  the  principles,  practices,  and  norms  already  established  under  AETC 
need  to  be  fully  implemented  to  ensure  that  DLIELC  becomes  more  responsive  and  account¬ 
able  to  DoD  policies  and  priorities. 


CHAPTER  TEN 


Summary  of  Key  Findings  and  Recommendations  and 
Implementation  Plan 


Summary  of  Key  Findings 

Based  on  our  analysis,  we  found  eight  critical  management  areas  that  have  problems  that  need 
to  be  resolved.  They  can  be  thought  of  in  terms  of  the  elements  of  effective  organizations  to 
which  they  correspond.  From  our  research  and  analysis  of  DLIELC’s  mission  and  perfor¬ 
mance,  we  determined  the  following  important  challenges  in  these  eight  areas: 

1.  Policy.  DLIELC  is  governed  by  a  maze  of  guidance.  However,  the  guidance  does  not 
necessarily  reflect  the  reality  of  the  circumstances  within  which  it  must  operate. 

2.  Business  Model/Requirements.  DLIELC  lacks  a  robust  requirements  determination 
process,  and  the  organization’s  business  model  is  unable  to  meet  variable  demand. 

3.  Financial  Risk.  Current  financial  management  practices  expose  ELT  clients  (defined 
as  DoD,  Department  of  State,  and  foreign  countries)  to  needless  financial  risk. 

4.  Technology.  DLIELC  should  introduce  new  technologies  at  a  quicker  rate.  The  suc¬ 
cessful  use  of  these  technologies  could  lower  fixed  and  variable  costs. 

5.  Identity.  DLIELC  has  conflicting  priorities,  which  have  led  to  contradictory  views 
on  the  institution’s  identity  (i.e.,  is  it  an  academic,  military,  or  government/policy 
institution?). 

6.  Assessment.  The  lack  of  a  formal  and  functioning  assessment/evaluation  process  inhib¬ 
its  DLIELC  from  being  able  to  define  and  assess  its  own  effectiveness. 

7.  Manpower.  The  hiring  process  (lasting  six-plus  months)  is  unable  to  respond  quickly  to 
short-term  and  cyclical  demand. 

8.  Organization/Advocacy.  Importantly,  the  organization  has  had  no  clear  institutional 
advocate,  resulting  in  a  lack  of  focused  oversight  within  the  Department  of  Defense  as 
it  relates  to  ensuring  the  effective  and  efficient  execution  of  ELT.1 


1  During  the  course  of  this  study,  HQ  AF/A1DG  took  on  this  tasking.  The  matter  is  still  worth  noting  until  the  evidence 
is  gathered  regarding  the  effectiveness  of  this  arrangement. 
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Summary  of  Key  Recommendations 

To  address  these  eight  critical  management  areas,  we  identified  recommendations  for  each 
area,  which  are  captured  in  Table  10.1.  They  are  discussed  in  more  detail  in  the  rest  of  the 
document. 

Table  10.1 
Recommendations 


Critical  Area 

Recommendations 

Policy 

Fully  review  all  applicable  rules  and  regulations  to  ensure  currency  and  appropriateness 

Create  two  instructions,  a  governing  DoD  instruction  and  a  consolidated  AF  instruction, 
covering  everything  needed  to  grade,  audit,  govern,  cost,  and  direct  the  organization 

Consider  leveraging  existing  AETC  processes  and  procedures  where  they  may  create 
opportunities  to  fully  tap  into  AETC's  resources  and  expertise 

Review  existing  rules  and  regulations  to  look  for  underutilized  authorities  and  provisions  that 
could  be  exercised  to  enhance  DLlELC's  effectiveness 

Business  model/ 
requirements 

Institute  a  process  to  enable  more  accurate  depiction  of  demand  using  the  proposed  model 
Reduce  seasonal  peaks  by  delaying  certain  categories  of  students 

Improve  utilization  of  existing  instructors  by  restricting  leave  during  peak  season 

Create  "breathing  space"  within  existing  supply  via  possible  relocation  of  U.S.  Army  students 
or  by  the  creation  of  a  "finishing  class" 

Increase  supply  by  utilizing  proposed  supply  strategy 

Financial  risk 

Form  a  task  force  of  senior  leadership  from  the  37th  Training  Wing,  HQ  AETC,  and  DLIELC  to 
implement  cost  recovery  reform 

Have  the  task  force  review  proposed  costing  models  to  ensure  methodological  accuracy 

Once  validated,  fully  cost  DLlELC's  activities 

After  costing  library  is  developed,  review  and  reissue  financial  management  policies  and 
procedures  as  they  relate  to  DLIELC 

Technology 

Explore  where  there  may  be  cost-effective  means  to  deliver  some  ELT  through  distance 
learning 

Explore  computerized  methods,  such  as  OPIc® 

Consider  using  a  collaborative  platform  approach  to  update  SET  textbooks 

Provide  all  students  (not  just  SET  students)  with  laptops 

Buy  additional  commercial  bandwidth  for  Internet  usage;  add  costs  to  fixed  cost  of  operating 
schoolhouse 

Identity 

Deemphasize  the  academic  character  of  the  organization 

Look  beyond  ELT  credentials  in  teacher  recruitment  process 

Have  more  military  personnel  interact  with  IMSs 

Consider  creating  an  alumni  program 

Consider  involving  IMSs  in  self-assessment  for  placement,  progression,  or  curriculum 
Collaborate  with  Defense  Language  Testing  Advisory  Board  to  fully  review  the  validity  and 
reliability  of  the  ECL 

Expand  faculty  PST  to  provide  basic  skills  and  understanding  in  security  cooperation 
Reemphasize  and  develop  field  trips  and  cultural  activities 
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Table  10.1 — Continued 


Critical  Area  Recommendations 

Assessment  Leverage  AETC's  existing  training  assessment  process  including  conformance  to  training 

development  and  evaluation  policies 

Explore  standard  processes  for  training  development  and  assessment,  gaining  access  to  lessons 
from  similar  training  organizations 

Improve  assessment  of  the  FSP  and  coordinate  with  the  Air  Force  Culture  and  Language  Center 
to  develop  measures  for  assessing  knowledge  and  comprehension  of  U.S.  culture 

Reexamine  balanced  scorecard  to  fully  represent  each  of  the  four  performance  measure 
perspectives 

Reevaluate  current  metrics  to  ensure  they  are  clearly  written;  relevant;  inclusive  of  academic, 
military,  and  building  partnerships  (BP)  objectives;  and  primarily  under  DLlELC's  control 

Include  specific  metrics  geared  toward  measuring  the  long-term  BP  impact 


Manpower  Utilize  U.S.  Code,  Title  10,  civilian  academic  hires  provisions 

End  the  current  practice  of  curriculum  development,  which  unnecessarily  consumes  faculty 
manpower 

Establish  a  dedicated  contracting  vehicle  to  purchase  English  language  instruction  and  support 

Have  specialized  military/technical  language  curricula  requirements  executed  by 
knowledgeable  experts  in  those  fields 

Emphasize  the  importance  of  building  partnerships  by  obtaining  more  military  department 
personnel 

Use  experts  and  consultants  as  a  ready  pool  of  language  professionals  who  could  be  brought 
on  to  quickly  fill  gaps  because  of  unforeseen  student  load  fluctuations 


Organization/  Ensure  that  OSD  and  USAF  leadership  agree  on  clarified  oversight  responsibilities  and 
advocacy  functions  for  DLIELC 

Ensure  that  an  advocate  or  champion  in  DoD  is  actively  involved  in  the  requirements  process 

Fully  implement  principles,  practices,  and  norms  established  by  AETC  to  ensure  DLIELC 
becomes  more  responsive  and  accountable  to  DoD  policies  and  priorities 


Implementation  Plan 

OSD  and  AF/A1  asked  RAND  to  help  DLIELC  in  its  strategic  planning.  The  RAND  team 
took  this  to  mean  assisting  OSD,  AF/Al,  and  DLIELC  to  develop  an  integrated  plan  to  imple¬ 
ment  the  findings  of  this  report.  Given  the  extensive  nature  of  the  findings  of  this  report,  as 
well  as  the  large  number  of  recommendations  to  address  those  findings,  we  outlined  how  those 
recommendations  can  be  implemented. 

In  Appendix  C,  we  include  a  matrix  we  developed  with  a  view  toward  facilitating  the 
implementation  of  the  recommendations.  The  plan  has  been  organized  to  represent  the  key 
recommendations  of  the  report  in  13  outcomes.  These  outcomes  are  organized  in  the  same 
fashion  as  they  appear  in  the  report.  These  are  to  be  achieved  by  the  various  supporting  inputs 
articulated  in  the  report.  Each  outcome  and  input  also  refers  to  the  appropriate  implementing 
agencies  (see  Appendix  C).  Also,  we  distinguish  where  each  outcome  and  input  touch  one  of 
the  eight  management  areas  of  this  report. 

Finally,  each  of  the  13  outcomes  has  been  assigned  a  priority  or  sequencing — immediate 
(now),  mid-term  (within  two  to  five  years),  and  long-term.  These  time  dimensions  represent 
both  implementation  timelines  and  their  importance  and  the  immediacy  of  each  outcome  to 
improving  DLlELC’s  financial  efficiencies  and  operational  effectiveness. 

In  Table  10.2,  we  summarize  the  13  key  outcomes  in  terms  of  the  management  area  and 
priority  (which  does  not  exclude  execution  in  tandem  or  imply  sequentiality). 
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Table  10.2 

Recommendations  to  Address  Problems  in  the  Critical  Areas  Identified 


Outcome  (Critical  Area) 

Priority 

1.  Clarified  mission  statement/priorities  in  conformance  with  USAF,  DSCA,  OSD  policies  and 
priorities  (M/P;  O/A) 

2.  Enhanced  policies/procedures,  implemented  to  increase  managerial  effectiveness  (M/P;  O/A) 

3.  Policy  to  improve  prediction  of  ELT  demand  (R;  1;  M) 

4.  Policy  to  prioritize  student  flow  by  reducing  seasonal  demand  peaks  (R) 

5.  Policy  that  expands  teaching  capacity  by  existing  labor  supply  and  that  introduces  greater 
flexibility  in  managing  labor  to  meet  peak  demand  periods  (FM/M) 

6.  Policy  to  manage  increase  in  supply  of  ELT  (R) 

7.  Policy  that  overhauls  existing  management  of  finance  system,  practices,  and  key  supporting 
tools  (MP;  FM) 

8.  Policy  that  establishes  that  curriculum  development  will  be  undertaken  only  when 
reimbursable  (FM;  1;  M) 

9.  Policy  that  better  exploits  technology  in  support  of  ELT,  with  clear  objective  of  reducing 
costs  wherever  possible  (FM;  T;  M) 

10.  Policy  that  clearly  establishes  that  DLIELC  is  ELT  organization  and  essential  BPC  instrument 
(1;  O/A) 

11.  Effective  assessment  process  (A) 

12.  Policy  to  expand  DLlELC's  labor  flexibility  (R;  FM;  1;  M) 

13.  Policy  that  seeks  to  improve  advocacy  for  DLIELC  by  senior  sponsors  (O/A) 

Immediate 

Immediate 

Immediate 

Mid-term 

Mid-term 

Long-term 

Immediate 

Immediate 

Long-term 

Long-term 

Mid-term 

Long-term 

Mid-term 

NOTES:  M/P  =  Mission/Policy;  R  =  Requirements;  FM  =  Financial  Management;  T  =  Technology;  I  =  Identity; 

M  =  Manpower;  O/A  =  Organization  and  Advocacy.  Immediate  refers  to  execution  years.  Mid-term  refers  to  the 
Future  Years  Defense  Program  (FYDP).  Long-term  refers  to  FYDP  and  beyond. 
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Measuring  Partnership  Building 


Table  A.1 

Potential  Application  to  DLIELC  of  Public  Diplomacy  Assessment  Metrics3 

BP  Objective 

Proposed  Public  Diplomacy 
Assessment  Metrics 

Potential  Application 
to  DLIELC 

Increase  understanding 
of  U.S.  policy  and  culture 

Monitor  "exit  survey  responses  that 
demonstrate  comprehension  of  U.S. 
culture  following  a  cultural  event" 
and  follow  evolution  from  one  FY  to 
the  next 

Improve  surveys  administered 
to  students  after  field  trips  and 
aggregate  results  to  observe  successes, 
failures,  and  trends 

Measure  "relative  change  in  Fulbright 
scholars'  knowledge  of  the  U.S.  after 
Fulbright  exchange"  by  comparing  pre- 
and  post-exchange  surveys 

Create  a  pre-  and  post-training  survey 
and/or  quiz  focusing  on  U.S.  culture 
and  values,  and  monitor  pre-  and  post¬ 
score  improvement 

Measure  "Overall  score  of  general 
population  focus  group  respondents 
concerning  the  improvement  of  their 
understanding  of  U.S.  culture" 

Hold  focus  groups  with  students  (e.g., 
students  who  have  completed  their  ELT 
and  are  waiting  for  their  FOT  to  start) 
to  find  out  how  their  knowledge  and 
perceptions  of  the  United  States  has 
changed 

Increase  favorable  opinion 
toward  the  U.S. 

Not  easily  transposable  to  the  DLIELC  case 

Increase  U.S.  influence  in 
the  world 

Examine  "relative  change  in  opinion 
polls  of  students  who  have  participated 
in  educational  exchanges  with  the 

United  States  which  state  their  level  of 
satisfaction  with  their  experience" 

Monitor  students'  level  of  satisfaction 
over  time,  as  expressed  in  surveys 
(DLIELC  already  collects,  aggregates, 
and  analyzes  these  results) 

a  Kenneth  Matwiczak,  project  director,  Public  Diplomacy  Model  for  the  Assessment  of  Performance,  report  to 
the  U.S.  Advisory  Commission  on  Public  Diplomacy,  Lyndon  B.  Johnson  School  of  Public  Affairs,  University  of 
Texas  at  Austin,  September  2010,  pp.  82,  87,  96,  117. 
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APPENDIX  B 


Cost  Model 


Description  of  Cost  Model  Tables 

Table  B.l  includes  all  direct  costs  associated  with  student  training.  It  is  a  roll-up  of  calculations 
from  Tables  B.3,  B.4,  B.5,  and  B.6.  Table  B.2  adds  the  costs  in  Table  B.l  to  indirect  costs  from 
calculations  in  Table  B.7  to  compute  the  total  cost  of  training.  The  costs  represented  in  these 
tables  are  best  estimates  and  are  intended  to  represent  a  methodology  in  approaching  costs.  The 
costs  factors  are  not  validated.  We  propose  the  model  as  a  methodology  for  computing  total 
costs,  not  as  the  final  answer.  The  tables  list  the  direct  cost  items  that  should  be  accounted  for 
in  the  program  in  four  major  categories.  It  remains  for  the  Air  Force  to  validate  the  costs  with 
official  estimates.  RAND’s  intent  was  to  produce  a  working  model,  not  to  give  a  final  number 
for  the  cost  of  producing  English  language  training.  Having  said  that,  our  estimate  of  $54.5 
million  is  considerably  higher  than  the  actual  POM  costs  of  $44  million. 


Table  B.l 

Roll-Up  of  Direct  Student  Costs 


Summary  of  Direct  Student  Costs 

Cost  per 
Student 

Yearly  Cost  (all  students) 

Phase  1:  In-processing  (one-time  cost/student).  See  Table  B.3 

$16.45 

$52,939 

Phase  2:  Fixed  direct  costs  (weekly  cost/ADSL).  See  Table  B.4 

$789.65 

$48,809,078 

Phase  2:  Variable  direct  costs  (weekly  cost/ADSL).  See  Table  B.S 

$32.53 

$2,010,773 

Phase  3  -  Out-processing  (one-time  cost/student).  See  Table  B.6 

$65.80 

$211,743 

Total  direct  costs  (all  students) 

$51,084,533 

NOTE:  ADSL  =  average  daily  student  load. 


Table  B.2 

Estimate  of  Direct  and  Indirect  Student  Costs 

Estimate  of  All  Costs 

Estimate  (yearly) 

Direct  costs.  See  Table  B.l 

$51,084,533 

Indirect  costs  (w/o  bldgs.)  See  Table  B.7 

$3,381,375 

Total 

$54,465,908 
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Table  B.3 

Phase  1:  Direct  In-Processing  Costs  per  Student 


Description 

Detail 

Total: 

First  Week 
Prior 
to  Class 

Divide  Total 
Cost  by  Entire 
Population? 

In-processing  schedule  (Thurs.  and  Fri.  before  first  day  of  class  and  finance  brief  on  Mon.  at  end  of  first  day 
of  class) 

Roll  call  (Thurs.) 

1  GS-07  for  30  min. 

$25.13 

YES 

Alumni  briefing  (Thurs.) 

1  GS-07  for  15  min. 

$12.56 

YES 

ID  card  briefing  (Thurs.) 

1  E-6  for  15  min. 

$10.82 

YES 

Roll  call — Medical  packets  and  dependent  insurance 
checks  (Thurs.) 

1  GS-07  for  25  min. 

$10.07 

YES 

ECL  video  and  tutorial  (Thurs.) 

1  GS-07  for  30  min. 

$12.08 

YES 

Testing  briefing  (Thurs.) 

1  GS-11  and  1  GS-05 
for  30  min. 

$27.63 

YES 

Roll  call  (Fri.) 

1  GS-07  for  15  min. 

$6.04 

YES 

Field  Studies  Program  briefing  (Fri.) 

2  GS-09s  for  1  hr.  each 

$59.11 

YES 

CPM  briefing  (Fri.) 

1  GS-11  for  30  min. 

$17.88 

YES 

Information  technology  briefing  (Fri.) 

1  GS-1 1  for  20  min. 

$11.92 

YES 

37  TRW/JA  (Fri.) 

1  0-3  for  15  min. 

$16.11 

YES 

Dean  of  Academics  (Fri.) 

1  0-5  for  15  min. 

$22.32 

YES 

Commandant  of  Troops  (Fri.) 

1  0-5  for  15  min. 

$22.32 

YES 

MTM  briefing  (Fri.) 

1  E-5  for  1  hr.,  20  min. 

$49.63 

YES 

Commandant/Deputy  Commandant  welcome  (Fri.) 

1  GS-15  for  20  min. 

$23.61 

YES 

Physical  training  briefing  (Fri.) 

1  GS-09  for  10  min. 

$4.93 

YES 

EM  (Fri.) 

1  GS-09  for  15  min. 

$7.39 

YES 

Finance  briefing  (Mon.) 

3  people  (E3,  E5, 

GS-09)  from  802 
finance  hold  finance 
brief  for  4  hours 

$372.78 

YES 

Manning  for  Arrival  Activities  Students  Must  Complete 

YES 

Testing 

45  min.  in  computer 
lab — Additional  GS-11 
lab  technician  present 

$26.82 

YES 

ID  card 

GS-05,  8-hr.  shift:  CAC, 
fingerprints,  PIN 

$156.03 

YES 

IT/automation 

2  GS-09s  for  2  hrs.  each 

$  18.21 

YES 

BOS  Costs  (Thurs. -Sun.  prior  to  first  day  of  class  Monday) 

Meals 

$11/day  per  student 

$44.00 

NO 

Lodging 

$39/day  per  student 

$156.00 

NO 

Incidentals 

$10.80/day  per  student 

$43.20 

NO 

Total  manpower-associated  costs,  all  units,  per  student 

$16.38 

Total  manpower-associated  costs  only,  DLIELC  only,  per  student 

$10.27 

Subtract  from 
Indirect  O/H 

Total  in-processing  cost  per  student  (first  four  days) 

$16.45 

NOTES:  O/H  =  overhead;  ID  identification  card;  ECL  = 

English  comprehension 

level;  CPM  =  country  program 

manager;  MTM  =  military  training  manager;  EM  =  emergency  management;  CAC  =  common  access  card;  PIN  = 
personal  identification  number. 


Table  B.4 

Phase  2:  Direct  Fixed  Costs  per  Student 


Description  Detail 


Manpower  (salaries) 

Instructors 

GS-lls.  Based  on  average  daily  student  load  and 
student:instructor  ratio 

Training  instructors 

GS-lls.  Uses  number  of  training  instructors  from  earlier 
input.  Training  instructors  may  or  may 
not  be  directly  involved  with  students  depending 
on  what  they  are  learning.  50%  of  time  directly 
involved  with  students 

OPI  instructors 

GS-lls.  Uses  number  of  OPI  instructors  from 
earlier  input.  If  not  conducting  OPIs,  they  are  expected 
to  work  on  updating  their  OPI  materials  as  well  as 
completing  one  self-assessment  per  year 

Operations  team — manning 
front  desk 

3  GS-04s 

Crisis  team 

1  E-4,  1  E-8,  1  0-2,  1  0-4 

Bookshop 

MTMs 

1  GS-05  manning  shop,  8  hrs./day,  5  days/wk. 

Doing  accountability/discipline  issues;  5  MTMs  (E-5)  each 
spending  5  hrs./day 

Materials 

Commercial  texts  GPC 

In  CY2011,  $83,171.73  in  commercial  texts  and 

and  BPA  $35,000.00  Form  9 

Packing  supplies  and  materials  Total  purchases  on  GPC  was  $26,870.63 

Writing  instruments,  labels,  Total  cost  from  May  2010  to  April  2011  was 


paper,  etc. 

IT  Services 

$474,971.76 

Internet 

Time  Warner  monthly  cost  $11. 8K  per  year 

Server  printers 

13  server  printers  at  $3.5K  each.  Equip,  refresh 
rate  is  3  years 

Classroom  printers 

150  classroom  printers  at  $300  each.  Equip,  refresh 
rate  is  3  years 

Total  Divide  Total 

Classroom  Nonclassroom  Cost  by  Entire 

Costs  Costs  Population?  Notes 


$244,573.59 


YES 


$1,430.25 


Estimated  50%  of 
time  goes  toward 
nonteaching  training 


$4,290.76  Estimated  75%  of 

time  goes  toward  OPI 
material  development 


$2,129.42 

YES 

$8,715.69 

YES 

100%  of  their  time 
goes  toward  crisis 
management 

$780.16 

YES 

$4,653.28 

YES 

Estimate  of  hrs./ 
day  spent  doing 
accountability  work 

YES 

$2,272.53 

YES 

$516.74 

YES 

$9,134.07 

YES 

YES 

$226.92 

YES 

$291.67 

YES 

$288.46 

TRUE 
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Table  B.4 — Continued 


Description 

Detail 

Total 

Classroom 

Costs 

Nonclassroom 

Costs 

Divide  Total 
Cost  by  Entire 
Population? 

Notes 

PC  workstations 

245  PC  workstations  at  $880  each.  Equip,  refresh  rate 
is  3  years 

$1,382.05 

YES 

Servers 

10  edu.  servers  at  $3.3K  each.  Equip,  refresh  rate  is  3  years 

$211.54 

YES 

Maintenance 

IT  maintenance  on  computers/laptops,  $28K/year  and  5% 
directly  supports  students 

$26.92 

YES 

Base  Operations  Support  Costs  (should  be  standard  for  all  students  in  all  MASLs  (courses)) 

Food  stipend 

Student  receives  $1 1  per  day  stipend  for  food 

$77.00 

NO 

Lodging  stipend 

Student  receives  $39  per  day  stipend  for  lodging 

$273.00 

NO 

Incidentals  stipend 

Student  receives  $10.80  per  day  stipend  for  incidentals 

$75.60 

NO 

BOS  factor 

FY11  BOS  factor  at  $2, 000/student 

$38.46 

NO 

Lodging  energy  costs 

104  kw  individual  daily  avg.  at  $.045  cents  per  kw 

$32.76 

NO 

Not  clear  if  accounted 
for  in  BOS  factor 

Lodging,  water,  and  sewage 
costs 

55  gallons  a  day  at  $.04  per  gallon  per  individual 

$15.40 

NO 

Not  clear  if  accounted 
for  in  BOS  factor 

Facility  energy  costs 

104  kw  individual  daily  avg.  at  $.045  cents  per  kw 

$32.76 

NO 

Not  clear  if  accounted 
for  in  BOS  factor 

Facility  water/sewage  costs 

8  gallons  a  day  at  $.04  gallons  per  individual 

$2.24 

NO 

Not  clear  if  accounted 
for  in  BOS  factor 

Medical 

Medical  visits 

1  flight  surgeon,  1  nurse,  2  techs  each  at  8-hr.  shifts. 

$600k  for  total  package,  including  contracting  fee  and 
med  staff.  Rough  estimates:  3-5  sick  visits  per  day,  2-4 
physicals  per  day,  numerable  walk-ins.  Given  that  medical 
staff  is  widely  used  by  more  students  than  those  that 
require  physicals,  all  medical  costs  will  be  spread  to  each 
student 

$11,538.46 

YES 

Assume  DLIELC  pays 
medical  contract.  Not 
considered  a  manpower 
cost  to  DLIELC 

Total  manpower-only  weekly  cost  per  student  (excludes  instructor) 

$13.69 

Subtract  from  indirect 
O/H 

Total  fixed  class  and  nonclass  weekly  cost  per  student 

$216.99 

$572.67 

Total  fixed  weekly  costs  per  student  $789.65 

NOTES:  From  802nd  Mission  Support  Group:  FY96  BOS  Factor  =  $1, 371/student,  used  1.159  FY07  inflation  factor  to  obtain  FY07  BOS  Factor  =  $1, 589/student.  We  used 
Consumer  Price  Index  conversion  from  1996  to  2011  to  obtain  $2, 000/student. 
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Table  B.5 

Phase  3:  Direct  Variable  Costs  per  Student 


Description 

Detail 

Total 

Classroom 

Costs 

Nonclassroom 

Costs 

Remarks 

Student 

Population 

Involved 

Requirements  by  MASL  (course) 

Physical  training 

3  days/wk.,  one  hour  each,  2  GS-4s.  100  attended 
of  300  IMSs  required  to  attend  PT  last  year  so  will 
use  100  out  of  total  yearly  population 

$106.47 

100 

IT 

Laptops 

75%  of  laptops  go  to  student.  Roughly  250/yr. 
laptops  purchased  at  $1,100  each.  469  laptops  in 
total  inventory 

$5,288.46 

If  75%  of  laptops  go  to  students, 
then  352  students  receive  a  laptop. 
352/total  population  is  percentage 
of  students  who  receive  a  laptop  = 
3.4% 

352 

Software  for  laptops 

$15,000  additional  software  for  students 

$288.46 

If  75%  of  laptops  go  to  students 
then  352  students  receive  a  laptop. 
352/total  population  is  percentage 
of  students  who  receive  a  laptop  = 
3.4% 

352 

Testing 

Quizzes 

Weekly  for  GET,  biweekly  for  SET.  In  computer  lab. 
One  GS-11  lab  technician.  5  BALIC  in  27  wks,  5  SET 
in  9  wks,  2  ALPS1  in  12  wks,  and  then  weekly  GET 
quizzes.  Quiz  is  45  min.  in  length 

$492.04 

2010  numbers:  94  BALIC  students, 

767  SET  only  students,  1106  GET+SET 
students,  29  ALPS1  students,  1,014 
GET  only  students 

58 

TOEFL 

Taken  twice.  DLI  pays  the  testing  fee.  $170/test 

$340.00 

Cost  accounts  for  21  students/yr. 
who  take  TOEFL 

21 

Materials 

Texts/CDs  for  GET/SET 

In  CY2010,  $636,416.80  in  texts  and  CDs  for 

SET  and  GET  instructors  and  students.  90%  of 
students/instructors  attend/teach  GET/SET  (other 
10%  of  students/instructors  attend/teach  IDB) 

$12,238.78 

Cost  already  reflects  per  GET/SET 
student  per  week 

2,887 

Texts/CDs  for  IDB 

In  CY2010  texts  issued  to  IDB  by  course: 
BALIC=$8,6939.21,  ALPS=$3,314.51, 

AELIC=$1 6,048.94,  MACS=$6,643.27, 
TOEFL=$1,681.30,  MELT=$,2688.99, 

MDS=$1, 962.05 

$2,293.81 

Cost  already  reflects  per  IDB  student 
per  week 

331 

Cost  Model  75 


Table  B.5 — Continued 


Description 

Detail 

Total 

Classroom 

Costs 

Nonclassroom 

Costs 

Remarks 

Student 

Population 

Involved 

Field  Trips  and  Receptions 

AMIGO  Friday  Evening  Reception 

2  GS-7s  and  2  GS-9s  for  4  hours.  Each  reception 
costs  $8,000  in  catering.  Occurs  monthly 

$1,945.32 

250  staff  and  students  attending, 
assume  90%  of  attendees  are 
students 

2,700 

Field  studies 

Contract  cost  $700  per  bus  with  guide.  75% 
contract  bus,  25%  base  bus.  Avg.  40  students/bus. 

1  GS-9/bus  on  weekend  trip.  1  GS-11/bus  weekday 
class  trip.  104  weekend  trips/yr.  90%  one-bus 
trips.  104  weekday  trips/yr.  Already  subtracted  out 
Houston  weekend  trips 

$2,902.79 

Weekday  trips  already  factor  in 
normal  instructor  pay;  assuming 
zero  overtime.  Overtime  hours  for 
GS-09  were  normal  standard  pay 
rate,  not  1.5x  for  example 

9,152 

Trips  to  DC 

IDB  students  only.  1  GS-11  and  1  GS-9.  50  students/ 
trip.  5  trips/yr.  $80, 000/trip.  Comes  out  of  $35/ 
student/wk.  in  tuition 

$7,829.83 

Need  to  factor  in  TDY  cost  of  GS- 
11  and  GS-09.  Otherwise,  GS-11 
salary  already  covered  since  this  is 
a  M-F  trip.  GS-09  salary  currently 
accounted  for  in  computation 

250 

Overnight  Houston  trip 

2  GS-9s  24  hrs.  overtime  each.  100  students/trip 
$20, 000/trip.  12  trips/yr.  Included  in  104  weekend 
trips 

$4,942.74 

Overtime  hours  for  GS-09s  were 
normal  standard  pay  rate,  not  1.5x 
for  example 

1,200 

Total  manpower  variable  weekly  costs  attributed  to  DLIELC 

$6.95 

Subtract  from  indirect  O/H 

Total  variable  weekly  costs  per  ADSL 

$32.53 

NOTES:  ADSL  =  average  daily  student  load,  or  the  average  number  of  students  in  any  given  week.  If  a  curriculum  development  instructor  sits  in  the  class  while  a  new 
curriculum  is  being  tested,  that  individual's  billable  time  needs  to  be  captured.  ECL  testing  is  as  follows  (assuming  during  high  times):  20  instructors  (GS-11)  for  45 
minutes  (weekly  basis);  64  instructors  (GS-11)  for  45  minutes  (once  a  month). 
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Cost  Model  77 


Table  B.6 

Phase  4:  Direct  Out-Processing  Costs  per  Student 


Description 

Detail 

Weekly 

Totals 

Divide  Total 
Cost  by  Entire 
Population? 

Meals:  Thurs.  of  graduation  week 

$11/day 

$11.00 

NO 

Lodging:  Thurs.  of  graduation  week 

$3  9/d  ay 

$39.00 

NO 

Incidentals:  Thurs.  of  graduation  week 

$10.80/day 

$10.80 

NO 

Out-Processing:  IT/Automation 

2  GS-lls  18  hours  each  for  out-processing 
of  laptops,  including  re-imaging 

$1,287.23 

YES 

Administrative  processing  for 
graduation 

1  GS-7.  11.5  hrs./wk. 

$277.86 

YES 

Graduation  photo 

$4.20  per  photo.  Costs  include  everything 

$4.20 

NO 

Graduation  ceremony 

Graduation  ceremony  lasts  1  hr.  2 
presenters:  GS-12  and  0-6  each  needing 

2  hrs.  for  prep/delivery.  Graduation 
happens  weekly.  Additionally  3  E-6s 
help  for  4  hrs./wk.,  1  GS-11  for  2  hrs./wk., 
and  1  E-8  for  2  hrs./wk. 

$992.52 

YES 

Out-processing  briefing  1  GS-07  on  Tues.  of  graduation  week  for  $15.10  YES 

37.5  min.  (the  37.5  is  an  average  length  of 
time;  out-brief  can  take  60  min.  in  peak 
period,  30  min.  standard) 

Total  manpower  out-processing  costs  per  student,  DLlELC-only  $0.80  Subtract  from 

indirect  O/H 

Total  out-processing  (cost  per  student)  $65.80 
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Table  B.7 

Phase  5:  Indirect  Institutional  Overhead  Costs 


Overhead  Costs 

Manpower 

Detail  Costs  (yr.)  Double  Count 

Curriculum  development  manpower 

Curriculum  development,  materials/ 
maintenance 

10  GS-9s  +  3  GS-lls  $837,814 

Support  for  15  computers  +  3  copiers  $9,000 

+  binding  machine 

Curriculum  development  refresh 

Overhead  (front  office) 

AFSAT  portion  for  ELT  setup 

AETC/AI  portion  for  ELT  setup 

AETC/A5  portion  for  POM  preparation 

AMIGO  program  materials 

Administration/operations  materials 

Food  service  manpower 

Food  service  supplies 

BOS  manpower 

Base  operating  support  materials 

Replacement  costs 

3  year  refresh  $14,667 

1  Col,  3  Lt  Col,  2  Capt,  4  SSgt,  3  GS-05  $780,059  $(690,774) 

Automation,  paper,  ink,  etc.  $220,625 

Food  server  costs  for  AMIGO  Inn  $738,199 

Breakfast/lunch/dinner  costs  $165,000 

Building  cleaners/managers  $266,011 

Toilet  paper,  water,  cleaning  supplies  $250,000 

Computers/equipment  (3  yr.  refresh,  $100,000 

150  (library  +  staff)) 

Bldg,  refresh. 

25  bldgs.,  refresh  $500k  every  10  $1,250,000  Excluded 

years 

Depreciation 

Buildings  (50  years  replace,  25  bldgs.,  $2,500,000  Excluded 

$5.0  M  each) 

Total  fixed  costs  (w/o  bldg,  costs) 

$3,381,375 

Total  fixed  costs 

$7,131,375 

NOTES:  The  manpower  double  count  sums  up  previous  direct  costs  of  overhead  personnel  from  the  total  cost  of 
the  overhead.  The  remainder  of  the  overhead  is  indirect.  Building  costs  are  estimated  but  are  not  a  reimbursable 
item. 


APPENDIX  C 


Integrated  Implementation  Plan 


On  the  following  pages,  we  present  the  matrix  we  developed  to  facilitate  the  implementation 
of  our  recommendations.  The  plan  is  organized  to  represent  the  key  recommendations  of  the 
report  in  13  outcomes.  These  outcomes  are  organized  in  the  same  fashion  as  they  appear  in  the 
report. 
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Table  C.1 

Integrated  Implementation  Plan 


Organizational 

Change  Recommendations 
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Outcome  #1: 
Immediate 

Clarify  mission  statement  and  priorities  in  conformance  with 
USAF,  DSCA,  and  OSD  policy  and  priorities 

X 

X 

X 

Outcome  #2: 
Immediate 

Enhance  policies  and  procedures  and  implement  them  to 
increase  managerial  effectiveness 

X 

X 

X 

Input  #1 

Undertake  a  full  review  of  all  applicable  DLIELC  policies  and 
regulations  to  ensure  they  are  current  and  appropriate 

X 

X 

X 

Input  #2 

Adopt  existing  AETC  processes  and  procedures  when  they 
create  opportunities  to  use  AETC's  resources  and  expertise 

X 

X 

X 

X 

X 

Input  #3 

Review  existing  policies  and  regulations  to  find  underutilized 
authorities  and  provisions  that  can  enhance  DLlELC's 
effectiveness 

X 

X 

X 

X 

X 

Outcome  #3: 
Immediate 

Develop  a  new  policy  and  implement  it  to  improve  prediction 
of  ELT  demand  using  the  following  new  managerial  and 
planning  methods 

X 

X 

X 

X  X 

Input  #1 

Recalibrate  the  wheel 

X 

X 

Sub-input 

Identify  factors  that  affect  required  training  time,  such  as  native 
literacy  rate,  student  motivation,  and  other  factors  identified  in 
empirical  literature  on  learning  a  second  language 

X 

X 

Input  #2  Use  regression  models  developed  by  RAND  to  predict  student 

load  based  on  FMS 


X 


X 
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Table  C.1 — Continued 


Organizational 

Change  Recommendations 


o 

Q. 

c 

o 


c 

CD 

CD  >» 

g*  « 
2  > 


£ 

CD 

E 

0 

0  u 

E  2 

* 

> 

Oi 

_o 

>* 

£ 

0 

E 

0 

S 

u  O 

o 

1/1 

O 

£  »-* 

£ 

‘■P 

Ifl 

Q. 

CT 

0 

DC 

11 

U_  0 

_£ 

u 

|2 

£ 

0 

"O 

0 

< 

£ 

0 

■O 

£ 

0 

£ 

O 


--  05 
£  u 

0  O 

CT  > 
±:  T3 

O  0 


Input  #3 

Use  regression  models  developed  by  RAND  to  predict  average 

English  language  training  time  for  new  countries  based  on 
world  modernity  data.  Update  as  required 

Outcome  #4: 
Mid-term 

Develop  a  policy  to  prioritize  student  flow  by  reducing  seasonal 
demand  peaks  the  better  to  manage  better  resources  and  labor  X 
scheduling  through  considering  the  following  options: 

X 

X 

Option  #1 

Schedule  students  with  no  FOT  during  nonpeak  months 

X 

X 

X 

Option  #2 

Do  not  schedule  students  during  peak  months  who  will  attend  a 
frequently  occurring  FOT 

X 

X 

X 

X 

Option  #3 

Schedule  only  MTTs  deemed  critical  to  COCOM  TSCP  priorities 
during  peak  months  and  delay  other  MTTs  to  nonpeak  months 

X 

X 

X 

X 

Option  #4 

Prioritize  students  based  on  their  funding  source 

X 

X 

X 

Outcome  #5: 
Mid-term 

Develop  a  policy  that  expands  teaching  capacity  by  existing 
labor  supply  and  introducing  greater  flexibility  in  managing  X 

labor  to  meet  peak  demand  periods 

X 

X 

X 

Input  #1 

Increase  utilization  of  existing  instructors  by  restricting  leave  during 
peak  season 

X 

X 

Input  #2 

Explore  with  U.S.  Army  the  option  for  ECHO  Company  to  train  U.S. 
students  at  separate  U.S.  Army  locations  to  accommodate  different 

needs 

X 

X 

X  X 

X 

Input  #3 

Set  up  a  finishing  school  for  students  who  are  ECL  qualified/waiting 
for  SET/FOT  (no  class  size  limitation) 

X 

X 

X 
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Table  C.1 — Continued 


Organizational 

Change 


Recommendations 
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Sub-input 

Incorporate  military  jargon,  acronyms,  pace  of  speech,  accents,  etc. 

X 

X 

X 

Outcome  #6: 
Long-term 

Develop  a  policy  to  manage  an  increase  in  the  supply  of  ELT  ^ 

X 

X 

X 

Input  #1 

Increase  classroom  capacity  to  200  rooms 

X 

X 

X 

X 

Input  #2 

Increase  instructor  capacity  to  250  instructors 

X 

X 

X 

X 

Input  #3 

Prepare  for  execution  of  additional  surge  options 

X 

X 

Input  #4 

IDIQ  temporary  instructor  contractor 

X 

X 

X 

X 

X 

Input  #5 

Contracted  ELT  programs  (university  or  contractors) 

X 

X 

X 

Input  #6 

Temporary  facilities  rental  and  contracted  temporary  instructors 

X 

X 

X 

X 

X 

Outcome  #7: 
Immediate 

Develop  a  policy  that  overhauls  the  existing  management  of  the 
finance  system,  practices,  and  key  supporting  tools 

X 

X 

Input  #1 

Organize  a  joint  task  force  under  Director,  DLIELC  to  implement  cost 
recovery  reform 

X 

X 

X 

X 

Input  #2 

Review  RAND-developed  costing  models  and  validate  their  accuracy 

X 

X 

X 

Input  #3 

Review  is  to  be  informed  by  relevant  authorities  in  Title  10,  Title  22, 
and  USAF  regulations 

X 

X 

X 

X 

Input  #4 

Final  validated  models  need  to  conform  methodologically  with  FIQ 

AETC’s  models 

X 

X 

X 
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Table  C.1 — Continued 


Organizational 

Change  Recommendations 

DLI  internal  policy 

DLI  internal  management 

37th  TW 

HQ  AETC 

USAF,  DSCA,  OSD 

Mission/policy 

Requirements 

Financial  management 

and  cost  recovery 

Technology 

Identity 

Assessment 

Manpower 

Organization  and 

advocacy 

Input  #5  Develop  costing  library  capturing  all  costs  associated  with  ELT 

X 

X 

Input  #6  Joint  task  force  to  provide  decision  briefing  to  HQ  AETC,  USAF  A-1 , 

DSCA,  and  OSD  with  proposals  to  reform  the  financial  management 
policies  and  procedures  for  DLIELC 

X  XX 

X 

Outcome  #8:  Develop  a  policy  to  establish  that  curriculum  development  will  be 

Immediate  undertaken  only  when  it  is  reimbursable 

X  X 

XXX 

Input  #1  Develop  a  regulation  to  capture  all  costs  associated  with  curriculum 

development  to  be  invoiced  to  contracting  client 

X 

X 

Input  #2  Initiate  an  administrative  and  financial  process  to  procure  needed 

curriculum  from  commercial  sources 

X 

X  X 

(Implied)  input  Develop  a  policy  by  which  instructors  engaged  in  curriculum 

development  are  to  be  utilized  for  ELT 

X 

X  X 

Outcome  #9:  Develop  a  policy  to  better  exploit  technology  in  support  of  ELT,  with 

Long-term  the  c|ear  objective  to  reduce  costs  wherever  possible 

X 

XX  X 

Input  #1  Review  all  of  RAND's  technology  enhancement  recommendations 

and  develop  a  phased  implementation  plan  to: 

X 

X 

Sub-input  A  Use  technology  to  replace/reduce  time  for  in-country  MTTs 

X 

XX  X 
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Sub-input  B 

Explore  developing  a  collaborative  platform  approach  to  updating 

SET  textbooks 

X 

X 

X 

X 

X 

Sub-input  C 

Provide  all  students  with  laptops  and  add  cost  to  course  costs 

X 

X 

X 

X 

Sub-input  D 

Purchase  sufficient  commercially  available  bandwidth  necessary  for 
internet  usage;  costs  to  be  added  to  fixed  costs  of  all  ELT  courses 

X 

X 

Sub-input  E 

Explore  options  to  disconnect  students  from  the  NIPRNET 

X 

Outcome  #10: 
Long-term 

Draft  a  policy  that  clearly  establishes  that  DLIELC  is  an  English 
language  training  organization  and  an  essential  instrument  of  BPC  * 

X 

X 

X 

X 

Input  #1 

Develop  a  policy  to  state  that  DLIELC  is  not  an  academic 
organization 

X 

X 

X 

X 

X 

Input  #2 

Develop  BPC-relevant  criteria  for  hiring  instructors 

X 

X 

X 

X 

Input  #3 

Explore  ways  and  means  to  effect  greater  interaction  between  U.S. 
military  and  ELT  students 

X 

X 

X 

X 

Input  #4 

Explore  ways  and  means  to  create  an  alumni  association 

X 

X 

X 

Input  #5 

Consider  involving  IMSs  in  self-assessment  for  placement, 
progression,  or  curriculum 

X 

X 

X 
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Table  C.1 — Continued 


Organizational 

Change  Recommendations 

DLI  internal  policy 

DU  internal  management 

37th  TW 

HQ  AETC 

USAF,  DSCA,  OSD 

Mission/policy 

Requirements 

Financial  management 

and  cost  recovery 

Technology 

Identity 

Assessment 

Manpower 

Organization  and 

advocacy 

Input  #6  Collaborate  with  the  Defense  Language  Testing  Advisory  Board  to 

review  the  validity  and  reliability  of  the  ECL  to  ensure  that  students 
are  accurately  programmed,  placed  into  the  right  course  level,  and 
proficient  enough  to  attend  FOT 

X 

X  X 

Input  #7  Expand  faculty  PST  to  provide  basic  skills  and  understanding  in 

security  cooperation.  Providing  skills  and  understanding  in  key  areas 
related  to  the  core  mission  is  an  important  step  in  building  both 
mission  valence  (attractiveness  of  the  mission)  and  commitment  to 

the  school 

X 

XXX 

Input  #8  Reemphasize  and  develop  field  trips  and  cultural  activities 

X  X 

Sub-input  A  Evaluate  effectiveness  of  AMIGO  program 

XX  X 

Sub-input  B  Increase  funding  as  determined  effective 

X  X 

Outcome  #1 1 :  Develop  an  effective  assessment  process 

Mid-term 

X  XXX 

X  X 

Input  #1  Reexamine  balanced  scorecard  to  represent  fully  each  of  the  four 

performance  measure  perspectives 

X 

X 

Sub-input  Reevaluate  current  metrics  to  ensure  that  they  are  clearly  written; 

relevant;  inclusive  of  academic,  military,  and  BP  objectives;  and 
primarily  under  DLlELC’s  control 

X 

X  X 
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Table  C.1 — Continued 


Organizational 

Change 


Input  #2 


Input  #3 


Input  #4 


Input  #5 


Sub-input  A 


Recommendations 

Leverage  AETC's  existing  training  assessment  process,  to  include 
conformance  to  training  development  and  evaluation  policies,  such 
as  those  contained  in  AETC  Instruction  36-2201 ,  Technical  and  Basic 
Military  Training  Evaluation 

Explore  standard  processes  for  training  development  and 
assessment,  thereby  gaining  access  to  the  lessons  observed  by 
similar  training  organizations.  Conforming  to  existing  practices  will 
put  DLIELC  in  a  position  to  leverage  the  experience  and  resources  of 
the  full  AETC  staff,  including  the  AETC  Inspector  General 

Improve  assessment  of  the  FSP  and  coordinate  with  AF  Culture  and 
Language  Center  to  develop  measures  for  assessing  knowledge  and 
comprehension  of  U.S.  culture 

Include  specific  metrics  geared  toward  measuring  the  long-term  BP 
impact.  Assessing  long-term  BP  impact  is  difficult,  but  first  steps 
could  include: 

Set  up  a  pre-  and  post-training  survey  and/or  quiz  focusing  on  U.S. 
culture  and  values.  The  degree  to  which  students  improve  in  these 
domains  between  the  beginning  and  the  end  of  their  training  could 
be  matched  with  their  country  of  origin,  length  of  stay  at  DLIELC, 
initial  ECL  score,  course  attended,  and  the  specific  field  visits  made. 
Progress  could  be  monitored  over  the  years 


DLI  internal  policy 


DU  internal  management 
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Organizational 

Change  Recommendations 

DLI  internal  policy 

DLI  internal  management 

37th  TW 

HQ  AETC 

USAF,  DSCA,  OSD 

Mission/policy 

Requirements 

Financial  management 

and  cost  recovery 

Technology 

Identity 

Assessment 

Manpower 

Organization  and 

advocacy 

Sub-input  B  Hold  focus  groups  with  students  to  find  out  how  their  knowledge  and 

perceptions  of  the  United  States  have  changed  during  their  time 
at  DLIELC.  Insights  from  these  focus  groups  could  also  be  used  to 
refine  EOC  and  field  visit  surveys 

X 

X 

Outcome  #12:  Develop  a  policy  to  expand  DLlELC’s  labor  flexibility 

Long-term 

X  X 

XX  X  X 

Input  #1  Explore  all  possible  innovative  U.S.  government  hiring  practices  by 

initiating  a  policy  and  legal  review  to  ascertain  if  U.S.  Code,  Title  10, 
civilian  academic  hires  provision  applies  to  DLIELC;  if  so,  implement 
its  provisions  as  quickly  as  possible 

X  X 

XX  X  X 

Input  #2  Develop  a  dedicated  contracting  vehicle  to  purchase  English 

language  instruction  and  support.  HQ  AETC  leadership  and  DLIELC 
management  should  make  a  strong  case  for  the  unique  DLIELC 
requirement  for  a  robust  IDIQ  contract  vehicle  under  the  Federal 
Acquisition  Regulation  to  hire  ELT  instructors  and  curriculum 
development  for  ELT  instruction  and  curriculum  development,  as 
required. 

X  X  X  X 

XX  X  X 

Input  #3  Commission  only  those  experts  with  specialized  military/technical 

lexica  to  undertake  curricula  development  on  a  fully  reimbursable 

X 

XXX  X 

basis  invoiced  to  the  requesting  client 
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I  = 

QJ  (1) 


Organizational 

Change  Recommendations 

Input  #4  Enhance  BPC  by  exploring  ways  to  obtain  more  military  department 

personnel  as  instructors  on  a  cost-recovery  basis  with  the  military 

department.  Furthermore,  HQ  AETC  leadership  and  DLIELC  XX  X 

management  should  explore  hiring  retired  military  personnel  via  new, 
innovative  U.S.  government  hiring  practices,  or  on  contract 


Input  #5 

Consider  the  use  of  experts  and  consultants  as  a  ready  pool  of 
language  professionals  who  could  be  brought  on  to  quickly  fill  gaps 
due  to  unforeseen  student  load  fluctuations.  Moreover,  the  hiring 
of  part-time  instructors  could  add  a  credible  and  quickly  accessible 
“surge”  capability  without  detriment  to  other,  ongoing  administrative 
or  courseware  development  functions 

X 

X 

X 

Outcome  #13: 
Mid-term 

Develop  a  policy  that  seeks  to  improve  advocacy  for  DLIELC  by  ^ 

senior  sponsors 

X 

X 

X 

X 

Input  #1 

Ensure  Office  of  the  Secretary  of  Defense  and  U.S.  Air  Force 
leadership  agree  on  clarified  oversight  responsibilities  and  functions 

of  DLIELC 

X 

X 

X 

X 

X 

Input  #2 

Should  the  Office  of  the  Secretary  of  Defense  and  U.S.  Air  Force 
leadership  agree  that  DLIELC  should  remain  under  the  management 
of  the  U.S.  Air  Force,  the  principles,  practices,  and  norms  already 
established  under  AETC  need  to  be  fully  implemented  to  ensure  that 

X 

X 

X 

X 

X 

DLIELC  becomes  more  responsive  and  accountable  to  Department 
of  Defense  policies  and  priorities 
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Organizational 

Change  Recommendations 

DLI  internal  policy 

DU  internal  management 

37th  TW 

HQ  AETC 

USAF,  DSCA,  OSD 

Mission/policy 

Requirements 

Financial  management 

and  cost  recovery 

Technology 

Identity 

Assessment 

Manpower 

Organization  and 

advocacy 

Input  #3  Adopt  greater  managerial  flexibility  in  order  to  meet  variable  demand 

through  the  adoption  of  new  labor  delivery  options  and  a  thorough 
review  of  current  costing  methods  and  implementation  models  to 
ensure  that  all  costs  associated  with  English  language  training  are 
recovered  through  accurate  pricing  of  training 

X  XX 

XX  X  X 

Input  #4  Ensure  that  HQ  AETC  engages  itself  in  the  responsibility  for  DLIELC 

administration  and  evaluation 

X  X 

X  XX 
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